Triage-PiEvo: Evidence Admission for Principle-Evolving Scientific Agents

   Yingming Pu Affiliation: Westlake University Affiliation: Zhejiang University^*Equal contribution^†Corresponding author✉ lipeize05@kuaishou.com, {puyingming, lintao}@westlake.edu.cn    Xingguo Xu Affiliation: Kuaishou Technology Affiliation: Dalian University of Technology    Yiming Cui Affiliation: Kuaishou Technology Affiliation: Hangzhou Dianzi University    Yuansheng Gao Affiliation: Kuaishou Technology Affiliation: Zhejiang University^*Equal contribution^†Corresponding author✉ lipeize05@kuaishou.com, {puyingming, lintao}@westlake.edu.cn    Tao Lin Affiliation: Westlake University

Abstract

Large language model (LLM)-based AI Scientists advance scientific discovery, yet they suffer from inefficiencies due to noisy observations. Existing principle-evolving agents act on every anomaly, leaving its source unresolved: a new mechanism or an evaluation artifact, so invalid revisions bias evolution. To address this, we formulate revision as evidence admission: anomalies steer search immediately, but only independently verified evidence may rewrite persistent principles, and contradicted principles are retracted. We realize this in Triage-PiEvo , which pairs an anomaly-responsive search portfolio with a staged verification gate that alone controls what enters principle state. Across four scientific tasks with three backbones, Triage-PiEvo  (1) attains the highest average solution quality with +7.5%∼11.8% over the strongest baseline per backbone, matching the strongest baseline’s full-budget quality with one-third of the evaluation budget on Qwen3-235B; and additional stress tests and cross-harness deployment demonstrate that (2) in constructed stress tests, it admits none of the corrupted, hidden-source, or exploit evidence that ungated variants admit, while missing no planted mechanism, and (3) plugged unchanged into three external research harnesses, it cuts false evidence admissions by over 60% and raises verified scores by 31%∼43%.

^†† Preprint.

1 Introduction

Autonomous scientific agents now automate scientific research within a single closed loop (Lu et al., 2024; Yamada et al., 2025; Schmidgall et al., 2025; Gottweis et al., 2025). On the premise that true knowledge often lies outside the initial hypothesis space, principle-evolution agents (Pu et al., 2026; Pu et al., 2025) make every anomaly a decision in this loop: steer hypothesis search, or rewrite the persisted theory and explanation.

While surprise-driven revision (Pu et al., 2026) lets agents escape their initial hypothesis space, it treats the observation’s reliability as given. Consequently, the revision may degrade because it rests on a signal that cannot discriminate, trading false revisions against missed theory blindly. Nor do existing methods cover this gap: verification loops critique transient responses rather than persisted theory (Shinn et al., 2023; Madaan et al., 2023; Gou et al., 2024; Dhuliawala et al., 2024), and self-correction without external evidence is unreliable (Huang et al., 2024; Kamoi et al., 2024). Accordingly, community evaluations find that auto-research agents produce extensive artifacts yet seldom revise their internal theory under contradictory evidence (Ríos-García et al., 2026; Kirgis et al., 2026). This rigidity leads to significant inefficiencies: anomalies are read as evidence of science, corrupted feedback enters trusted scientific state.

These limitations reduce to three challenges: (a) one anomaly score has source ambiguity that cannot tell a new mechanism from corrupted or misspecified evaluations; (b) the role conflation of one history serves both search and principle acceptance, so unreliable observations can rewrite trusted principles; and (c) the revision irreversibility that accepted principles cannot be retracted under contradictory evidence.

To bridge this gap, we propose Triage , an evidence-admission layer that decouples anomaly detection from principle revision (Pu et al., 2026). Triage treats each anomaly as source-ambiguous and routes evidence along two separately logged paths: (a) observations steer candidate selection, while (b) verification records update per-channel reliability estimates. A principle revision is trusted only after it survives independent tests on unseen cases, as shown in Figure 1. We instantiate this in Triage-PiEvo , which pairs Triage  with Active Flow Expansion (De Santi et al., 2026): the portfolio allocates the evaluation budget across exploitation, frontier expansion, and mechanism-discriminating probes, while the evidence-admission decides which evidence alters principle state.

We evaluate Triage-PiEvo on four discovery tasks under three backbone LLMs. Empirical results demonstrate that Triage-PiEvo : (a) attains the highest performance, raising average solution quality from PiEvo ’s 62.87–70.35% to 79.63–83.37%; (b) admits no corrupted or exploit-region observations in constructed stress tests where gate-off variants admit every one, under evaluator corruption and proxy exploitation alike (the latter further raising held-out score by 0.209); (c) cuts false expansions on injected noise, surrogate failure, and non-generalizing local artifacts from rate of 1.0 to 0.0 in the same tests without missing any planted mechanism, and backtracks to the correct state after mechanism reversal; and (d) transfers unchanged to three external research harnesses, cutting false evidence admissions by over 60% and raising verified scores by 31%∼43%.

In summary, our contributions are:

1. (a)

   We introduce Evidence Admission as the decision layer of anomaly-driven agents for changing scientific state. Anomalies are treated as source-ambiguous, and revision is governed by asymmetric costs of false revision versus missed mechanism.

2. (b)

   We propose Triage-PiEvo , which realizes this layer through per-channel trust estimation, a principle lifecycle gated by independent held-out support.

3. (c)

   We conduct extensive evaluations across four scientific tasks and three backbones: Triage-PiEvo  raises average solution quality by +7.5%∼11.8% over the strongest baseline per backbone, and controlled corruption, hidden-source, and proxy-exploitation studies show that the gate rejects all unreliable evidence injected in these studies while missing no planted mechanism.

2 Related Work

Autonomous scientific agents and evolving state. Scientific agents automate growing shares of the research cycle, from end-to-end paper production and multi-stage pipelines (Lu et al., 2024; Yamada et al., 2025; Tang et al., 2026; Schmidgall et al., 2025; Gottweis et al., 2025) to tool-coupled domain execution and closed-loop laboratory discovery (Bran et al., 2023; Boiko et al., 2023; Ruan et al., 2024; Yang et al., 2025; Ghafarollahi and Buehler, 2025; Ding et al., 2025; Ghareeb et al., 2026; Narayanan et al., 2024; Laurent et al., 2024; Mitchener et al., 2025a; Skarlinski et al., 2024; Mitchener et al., 2025b). What persists between iterations there is an artifact, not a revisable scientific account. A narrower line makes the internal state editable: PiEvo  and PiFlow expand a principle space on conflicting observations (Pu et al., 2026; Pu et al., 2025), ATLAS selects mechanism-discriminating experiments (Éltető et al., 2026), self-revising frameworks log regime changes (Wang and Buehler, 2026), and evolving architectures such as EvoMaster (Zhu et al., 2026) and ADMET-EvO (Zhou et al., 2026) pursue cumulative adaptation across tasks, with agentic laboratory world models unifying hypothesis generation and state tracking (Fu et al., 2026). Editable state is also corruptible: without explicit evidence admission, anomalous or corrupted feedback rewrites persistent principles directly (Ríos-García et al., 2026; Kirgis et al., 2026). Triage  separates the history that steers search from the evidence authorized to alter scientific state.

Active experimentation under heterogeneous sources. In active learning and Bayesian optimization, one line decides where to spend evaluations (MacKay, 1992; Houlsby et al., 2011), constraining exploration for safety (Sui et al., 2015) or scalability (Eriksson et al., 2019). Another exploits heterogeneous costs, fidelities, and failure rates across sources (Swersky et al., 2013; Kandasamy et al., 2017; Poloczek et al., 2017; Ghoreishi and Allaire, 2019), extending to unreliable channels (Mikkola et al., 2023; Fan et al., 2024), quality-controlled loops (Harris et al., 2025), and out-of-distribution expansion (De Santi et al., 2026). Both decide which observation to acquire before it is run, so an anomalous outcome inherits whatever reliability the acquisition rule assumed. The difference is in what the decision protects: multi-source methods weight a noisy source inside a regression model, while the object at risk here is a persistent principle that a single admitted artifact can corrupt for the rest of the run. Our AFE portfolio takes the acquisition-side view for candidate generation; Triage  makes the post-observation decision of whether an anomaly may update that principle.

Verification, provenance, and agent monitoring. In agent systems, verification targets transient outputs through feedback recycling (Yao et al., 2022; Shinn et al., 2023; Madaan et al., 2023), tool-based checking (Gou et al., 2024), independent verification questions (Dhuliawala et al., 2024), and evidence accumulation (Zhang et al., 2026), though self-correction without external evidence is unreliable (Huang et al., 2024; Kamoi et al., 2024). Related work argues for monitoring agentic systems before trusting them (Boston et al., 2026) and for immutable provenance trails over agent-generated artifacts (Liu et al., 2026), while Sibyl (Rangelov, 2026) uses temporal backtesting as held-out evidence. Triage  carries these provenance and monitoring ideas to persistent principles: same-source agreement keeps observations search-active, acceptance requires independent held-out support, and contradictory held-out evidence forces retraction.

3 Methodology

Overview. Triage-PiEvo focuses on maintaining a revisable account of what observations mean. In this context, each experiment has two possible consequences: (a) guiding the next experiment, and (b) changing the principles used to interpret future experiments. Based on our intuition:

A surprising observation can steer hypothesizing long before it is reliable enough
to revise persistent scientific principles.

As shown in Figure 2, Triage-PiEvo  separates using an observation from believing its explanation: it keeps exploration responsive to new signals while requiring independent evidence before revising a principle. Triage-PiEvo  builds on the principle-hypothesis-testing cycle of PiEvo (Pu et al., 2026).

This separation is implemented as a feedback loop with three components: (a) an evidence-admission layer checks whether an unexpected outcome generalizes to held-out cases (Section 3.2), (b) a search policy proposes experiments from outcomes that have been admitted for exploration (Section 3.3), and (c) a principle manager then updates the principle state accordingly (Section 3.4). Two histories keep these roles apart: the primary history accumulates observations the search may act on, while the verification history records linked checks that alone can justify—or overturn—a principle revision.

3.1 Evidence Admission under Source Ambiguity

We define the objects on which Triage  operates, proceeding in three steps: the round-level state, the source-ambiguity model, and the decision objective that any runtime admission rule must respect.

Round state and the anomaly event. The agent interacts with an environment over rounds t=1,…,T, and each round produces exactly one record through three steps: (i) the agent proposes a testable candidate xt∈𝒳; (ii) an internal predictor—the Gaussian Process (GP) experts {ℳP}P∈𝒫t of PiEvo  (Pu et al., 2026), one likelihood model per principle record, fit on the history of admitted observations—issues a prediction y^t with predictive scale σt>0; (iii) an observation channel ct∈𝒞 (e.g., an evaluator or data source) evaluates xt and returns an outcome yt with evaluator’s metadata mt (e.g., configuration). The tuple (xt,y^t,yt,ct,mt) constitutes the record; how far yt deviates from what 𝒫t predicted determines what happens to it next, per Definition 3.1.

Definition 3.1 (Anomaly score and anomaly event).

The anomaly score of a record is its standardized residual st≔|yt−y^t|/σt. Given a threshold τanom fixed before evaluation, the record is ordinary if st≤τanom and an anomaly if st>τanom.

An ordinary record is admitted directly to the primary history and triggers no further action. An anomaly, by contrast, opens a diagnostic state: it is withheld from search scores and principle updates until the verification procedure of Section 3.2 resolves it.

The anomaly score is blind to its source. A large residual is ambiguous in origin. We make this explicit by attaching to each evaluation a source variable Zt taking values in

denoting measurement noise, device fault, execution failure, mismatch of the external surrogate evaluator, out-of-distribution input, distribution drift, and a genuine new mechanism. Exactly one source generates each evaluation, and only zmech justifies revision; the remaining artifact sources 𝒵art should leave the principle state untouched. Because st is a deterministic function of (yt,y^t,σt) and not of Zt, the anomaly score is uninformative about which source produced it; thresholding st alone therefore cannot separate a discovery from an artifact. The consequences of deciding wrongly are asymmetric, as Example 3.2 illustrates.

Example 3.2 (Two anomalies with identical scores).

Suppose two rounds produce the same large score st. In round t1, a corrupted evaluator channel (zfault) inflates the outcome of a mediocre candidate; revising 𝒫t on this record redirects every subsequent prediction toward an artifact, corrupting all future rounds. In round t2, a genuinely novel candidate (zmech) violates the current principles; retaining 𝒫t merely postpones the discovery until stronger evidence arrives. The first error propagates through persistent state; the second only costs time.

To address this asymmetry, we define the admission decision as choosing d∈{𝗋𝖾𝗍𝖺𝗂𝗇,𝗋𝖾𝗏𝗂𝗌𝖾} after observing an anomaly, with loss

where ℓFR and ℓFM price false revision and false retention, respectively, with ℓFR≫ℓFM per Example 3.2. Under this loss, the risk-optimal action is to revise only when the posterior q⁡(zmech∣yt,y^t,mt) exceeds ℓFR/(ℓFR+ℓFM)—a threshold close to one.

Remark. The reference model of Eq. (2) dictates three essential properties: (i) anomaly detection must trigger further evidence collection; (ii) the evidence required for revision must be independent of the channel ct; and (iii) revising persistent state must demand strictly stronger evidence than search use does.

3.2 Escalating from Repeats to Held-Out Probes

Because an anomaly may affect candidate search, principle proposal, or principle revision, Triage  applies three checks (C1∼C3) of increasing independence from the originating experiment channel ct, stopping at the first failure. Each check rules out one class of artifact sources in 𝒵art (see Eq. (1)) and unlocks exactly one privilege.

C1: Rules out transient faults; unlocks search use. Re-evaluating xt on ct yields vtct. Agreement, |yt−vtct|≤ϵrep, excludes transient noise and execution failures (znoise,zexec). This cannot authorize revision, since repeating on ct replicates whatever systematic bias ct carries.

C2: Rules out channel-specific corruption; unlocks revision proposals. Re-evaluating xt on an independent channel c′≠ct yields vtc′. Agreement, |vtct−vtc′|≤ϵind, excludes corruption specific to ct (zfault), so the anomaly may propose a principle and become provisional (see Section 3.4).

C3: Rules out non-generalizing artifacts; unlocks acceptance. Evaluation by an independent evaluator on held-out inputs excludes effects tied to the original evaluation context (zsurr,zood,zdrift), and is the only evidence the acceptance gate counts (see Section 3.4). A record failing any check is quarantined—excluded from both histories’ influence until resolved; a verified replacement may remain search-active but cannot update persistent state by itself.

Reconciliation outcomes update per-channel reliability, so that frequently contradicted channels are progressively excluded from verification duty. Each channel c carries a reliability estimate rc∼Beta⁡(αc,βc) with trust score pctrust=αc/(αc+βc); agreement increments αc and disagreement increments βc for every participating channel (see Algorithm 1).

Remark. Trust selects which channels may verify, but does not lower the standard of evidence. All thresholds are fixed before evaluation and are independent of pctrust, so no amount of accumulated trust lets a channel certify its own anomalies.

3.3 Proposing Candidates without Seeing the Gate

We use Active Flow Expansion (AFE) (De Santi et al., 2026) as a search component in Triage-PiEvo . It proposes candidates in three roles: best-region exploitation, frontier expansion, and mechanism-discriminating probes.

Scoring candidates. For each role r, AFE scores a candidate x∈𝒳 by four terms computed from the GP experts of Section 3.1, refit on the admitted primary history:

where μ^​(x) and σ^​(x) are the GP experts’ predicted outcome and predictive uncertainty, N⁡(x) is the distance of x from already evaluated inputs, and D⁡(x) is a descriptor-novelty measure. The role weights {wrμ,wrσ,wrn,wrd} are fixed a priori and differ across roles: exploitation weights μ^, expansion weights N and D, and probes weight σ^ in regions where competing principles disagree (Figure 3).

AFE reads admitted evidence; Triage  alone writes it. Two append-only histories keep search evidence and revision evidence apart: the primary history Htprim stores the records the search policy may act on, and the verification history Htver stores the C1–C3 checks (see Section 3.2), each linked to the primary record it concerns. “Outcome-blind” is then a one-way boundary over these histories: AFE reads only Htprim and candidate descriptors, outputs xt, and is forbidden from verification channels, quarantine flags, and gate state; Triage  alone writes back, entering the admitted, substituted, or quarantined outcome into Ht+1prim for the next refit of the GP experts—PiEvo ’s per-principle likelihood models p⁡(y∣h,P) (Pu et al., 2026), which we use unmodified. Unable to observe gate state, the search policy can never smuggle unverified evidence into persistent state; the only write path to what the agent believes is the acceptance gate of Section 3.4.

3.4 A Gated Principle Lifecycle with Reversible Revision

The final component determines how verified evidence changes persistent state. Each principle P is a record (parent⁡(P),E⁡(P),state⁡(P))—a pointer to the principle it refines, its linked evidence set, and a lifecycle state that advances through Proposed⟶Provisional⟶Accepted, with any state sent to Retracted upon contradiction. We specify the two forward transitions and the retraction rule in turn.

From proposal to acceptance: only admissible evidence counts. An anomalous record passing C1–C2 (see Section 3.2) creates or updates a Proposed principle, and C2’s cross-channel agreement promotes it to Provisional. Acceptance, however, is gated by the following class of evidence:

Definition 3.3 (Admissible evidence for revision).

A record e is admissible for principle P if (i) it is produced by an evaluator process sharing no memory, cache, or random seed with the generator (independence), and (ii) it is measured on Xheldout disjoint from the generator-visible Xtrain (held-out). An admissible record supports P if its held-out gain Δheldout​(e,P)≥τgen and contradicts P if Δheldout​(e,P)≤−τgen; records in between are uninformative.

Writing nP+ and nP− for the numbers of supporting and contradicting admissible records in E⁡(P), Provisional⟶Accepted⟺nP−=0∧nP+≥k. The two conditions of Definition 3.3 each remove one failure mode: independence prevents corroboration from being manufactured by the proposing channel, and the held-out requirement makes acceptance test generalization rather than fit.

Retraction makes revision reversible. A single contradicting admissible record (nP−≥1) moves any active principle—Proposed, Provisional, or Accepted—to Retracted, and the retraction propagates through parent⁡(⋅), so descendants of a contradicted principle cannot remain trusted. Retraction is standing rather than proposal-time only: every newly admitted record is scored against each active principle it bears on, so evidence arriving after acceptance can still overturn an Accepted principle (Algorithm 1).

In summary, the full loop of Triage-PiEvo , i.e., AFE proposal, evaluation, staged verification (C1–C3), trust update, and lifecycle transition, is summarized in Algorithm 1, with thresholds and sensitivity analyses in Appendix A.

4 Experiments

We evaluate Triage-PiEvo  along four axes: (1) matched downstream performance across four scientific discovery tasks under three backbone LLMs; (2) search efficiency in optimization speed, candidate coverage, and computational cost; (3) controlled stress tests of the admission gate under evaluator corruption, hidden anomaly sources, mechanism reversal, proxy exploitation, and cross-harness deployment; and (4) ablations isolating search expansion from evidence admission.

4.1 Experiment Setup

Tasks and Benchmarks. Following Pu et al. (2026), we evaluate Triage-PiEvo  on four optimization tasks: Molecular Bio-activity (MBO), Nanohelix (NHO), Superconductor (SPO), and Transition Metal Complex (TMC). Each task is scored by its domain evaluator and maximizes a task-specific objective under a shared budget of 24 evaluations.

Baselines. We evaluate three categories of scientific agents : (a) generic and classic: Vanilla MAS and ReAct (Yao et al., 2022), which represent tool-use loops without persistent scientific state; (b) autonomous research: AI Scientist v1 (Lu et al., 2024), AI Scientist v2 (Yamada et al., 2025), and AI Researcher (Tang et al., 2026), which automate multi-stage pipelines over transient artifacts; and (c) principle-evolving: PiFlow (Pu et al., 2025) and PiEvo  (Pu et al., 2026), the state of the art that revises a principle set by anomalous—precisely the response our method re-governs.

Implementation Details. We evaluate three backbones: Qwen3-235B (Team, 2025), Gemini-2.5-Flash-Lite (Google, 2025), GPT-5.6-Luna (OpenAI, 2026) without extended chain-of-thought, over three fixed seeds, reported as mean ± std. Gate parameters in Triage-PiEvo are fixed a priori across all tasks and backbones. Full specification and sensitivity sweeps in Appendix A.

4.2 Evaluation Metrics

With y1:T the primary outcomes of a run, we employ metrics that capture abilities to (a) reach high-quality solutions, (b) reach them early, and (c) cover the candidate space broadly:

a. Solution Quality (SQ, %, ↑). The best outcome found, normalized by a published per-task reference μref from PiEvo  (Pu et al., 2026): SQ=100⋅maxt⁡yt/μref, with μref=6.5 (MBO), 2.0 (NHO), 298.5 (SPO), and 493.8 (TMC). These references are not attainable upper bounds, so SQ passes 100 when a run beats the reference; we apply no clipping.

b. Area Under the Optimization Curve (AUOC, %, ↑). The budget-averaged best-so-far quality, AUOC=100⋅1T∑tmaxt′≤t⁡yt′/μref, crediting how early quality is reached.

c. Average Pairwise Distance (APD, ↑). The mean pairwise cosine distance between proposed candidates. A higher APD indicates broader candidate coverage.

4.3 Performance and Efficiency Analysis

Table 1 compares Triage-PiEvo  against seven baselines on solution quality (SQ) across four tasks and three backbones, and Table 2 reports the corresponding coverage (APD) and accumulation speed (AUOC). Our analysis yields three primary observations:

Obs.➊: Triage-PiEvo attains the highest average SQ under every backbone. Triage-PiEvo  attains the highest SQ in every task–backbone and the highest average under every backbone, improving on the strongest baseline of each backbone by 11.75%, 11.14%, and 7.48% (Table 1), indicating that evidence admission repairs a failure mode shared across three backbones rather than one tuned to a single model.

Obs.➋: Triage-PiEvo  leads on average coverage and on accumulation speed. Triage-PiEvo achieves a 1.16×–1.22× faster accumulation (Table 2), while baselines trade the two facets off (e.g., AI Researcher (Tang et al., 2026) reaches 89.21 AUOC on MBO but at the lowest average coverage); on NHO, ReAct and AI Scientist v2 attain higher coverage but lower AUOC, so the lead holds on average rather than in every task–metric cell. This confirms that Triage-PiEvo filters evidence for revision without constricting the search.

Obs.➌: Triage-PiEvo opens its margin early and holds it at a bounded cost. On Qwen3-235B, Triage-PiEvo buys the strongest baseline’s full-budget result using one-third of the budget (Figure 4; Appendix B, Figure 7). The gate adds only +13.6% wall-clock time over PiEvo  (134.5s vs. 118.4s per run), while the multi-agent and tree-search frameworks it outperforms cost 1.28×–1.85× more (AI Scientist v2: 172.4s; AI Researcher: 188.2s; PiFlow: 248.6s), placing Triage-PiEvo  on the efficiency frontier of quality versus cost (Appendix C, Figure 8).

4.4 Stress Tests under Unreliable Feedback

We further intervene on the feedback channel and compare gated and ungated variants across two complementary regimes: (i) Constructed Stress Tests (Appendix E), where corruptions, proxy exploits, and distribution shifts are synthetically injected with deterministic ground truth; and (ii) Organic Benchmark Tasks (§4.5, Table 9), evaluating continuous, unlabelled noise. We use AutoResearchEval (Fei et al., 2026) for evaluation; see Appendix H for details. One observation carries the result:

Obs.➍: In the constructed stress tests, Triage-PiEvo rejects the unreliable evidence in every condition and loses no planted mechanism. Across evaluator corruption, hidden anomaly sources, and proxy exploitation, Triage-PiEvo  drives the ungated variants’ admission rate on unreliable evidence from 1.0 to 0.0—including exploits that reproduce perfectly—while recovering planted mechanisms with 1.0 recall and, in the proxy-exploitation study, improving held-out score by +0.209 (Tables 7 and 8 in Appendix E); the organic gate ablation (§4.5 and Table 9) traces this discrimination to the held-out criterion rather than repetition. The same gate acts in reverse and transfers: it flags a mechanism reversal at t=17, retracts the contradicted principle with its descendants one round later (t=18), and recovers the new optimum by t=23 where PiEvo  stagnates; attached unchanged to three external harnesses, it raises the externally verified score by 0.087–0.098 at additional token and runtime cost (Figure 5).

4.5 Ablation Study

We ablate at three levels: the candidate-generation and evidence-admission modules of Triage-PiEvo , each mechanism of the Triage  gate, and the gate’s decision thresholds.

Module Ablations. Table 3 reports a 48-cell factorial study on Qwen3-235B at budget 8. Triage-only falls below PiEvo  on SQ (60.60% vs. 64.78%): without search expansion, filtering only narrows an already narrow candidate flow, and the mild, unlabelled noise of the organic tasks gives it little to filter. Triage-PiEvo  w/o Gate restores exploration but admits whatever the evaluator reports. Triage  sets what may change persistent state, which is why the gate costs a little quality under routine noise and repays it under the injected unreliable feedback of §4.4.

Gate Mechanism Ablations. Ablating each component of the admission policy on organic benchmark tasks (Table 9, Appendix F) locates the protective effect in the held-out criterion: removing held-out verification raises false revision from 4.8% to 34.5%, and treating a same-source repeat as independent raises it to 45.8%—both admit evidence that is reproducible but not transferable, which is exactly the artifact case of Obs.➍. Removing channel reliability raises false revision to 19.2% through a different route: verification is then routed without regard to which channels have been reconcilable. The gate therefore depends on both the standard of proof and the choice of who supplies it. Mechanism recall here (92.4%) sits below the 100% of the constructed tests (Table 7): planted mechanisms carry effect sizes Δ≥2.5​σ0 that reliably clear τgen, whereas organic tasks also present weaker effects that can fall below it.

Hyperparameter Sensitivity. Sweeps over the gate thresholds (Appendix A, Figure 6) show the fixed operating point is not finely tuned: the default τgen=0.16 lies inside the zero-error region, tolerating over ∼80% held-out signal attenuation before a true mechanism is rejected, and the default τanom=0.85 sits on a flat zero-miss plateau before coverage collapses at more restrictive thresholds.

5 Conclusion

An anomalous residual is source-ambiguous: new mechanisms and evaluation artifacts produce identical surprisal. Triage  separates the two uses of such evidence: observations steer search immediately, while persistent principles require independent held-out support and are retracted on held-out contradiction. Instantiated in Triage-PiEvo , this design attains the highest average solution quality across four tasks and three backbones, admits none of the constructed corrupted or exploit evidence while missing no planted mechanism, recovers from mechanism reversal, and transfers unchanged to three external harnesses—all at +13.6% runtime.

References

• Boiko et al. (2023) D. A. Boiko, R. MacKnight, B. Kline, and G. Gomes Autonomous chemical research with large language models. Nature 624 (7992), pp. 570–578. Cited by: §2.
• Boston et al. (2026) M. F. Boston, G. Hanson, E. Georgala, J. Hudgens, and H. Frase Monitoring agentic systems before they’re reliable. arXiv preprint arXiv:2606.02494. Cited by: §2.
• Bran et al. (2023) A. M. Bran, S. Cox, O. Schilter, C. Baldassari, A. D. White, and P. Schwaller Chemcrow: augmenting large-language models with chemistry tools. arXiv preprint arXiv:2304.05376. Cited by: §2.
• De Santi et al. (2026) R. De Santi, B. Lee, C. P. Jensen, K. Protopapas, S. Tang, C. Liu, P. Chatterjee, Y. Yue, and A. Krause Active flow expansion for out-of-distribution discovery: from theory to molecules. arXiv preprint arXiv:2606.08802. Cited by: §1, §2, §3.3.
• Dhuliawala et al. (2024) S. Dhuliawala, M. Komeili, J. Xu, R. Raileanu, X. Li, A. Celikyilmaz, and J. Weston Chain-of-verification reduces hallucination in large language models. In Findings of the association for computational linguistics: ACL 2024, pp. 3563–3578. Cited by: §1, §2.
• Ding et al. (2025) K. Ding, J. Yu, J. Huang, Y. Yang, Q. Zhang, and H. Chen SciToolAgent: a knowledge-graph-driven scientific agent for multitool integration. Nature Computational Science 5 (10), pp. 962–972. Cited by: §2.
• Éltető et al. (2026) N. Éltető, N. D. Daw, K. L. Stachenfeld, and K. J. Miller Atlas: active theory learning for automated science. arXiv preprint arXiv:2606.12386. Cited by: §2.
• Eriksson et al. (2019) D. Eriksson, M. Pearce, J. Gardner, R. D. Turner, and M. Poloczek Scalable global optimization via local bayesian optimization. Advances in neural information processing systems 32. Cited by: §2.
• Fan et al. (2024) M. Fan, B. Yoon, E. Dougherty, F. Alexander, N. Urban, R. Arróyave, and X. Qian Multi-fidelity bayesian optimization with multiple information sources of input-dependent fidelity. In The 40th Conference on Uncertainty in Artificial Intelligence, Cited by: §2.
• Fei et al. (2026) Y. Fei, N. Liu, X. Yu, S. Chen, L. Li, R. Thapa, M. Ciobanu, Q. Mao, and R. Das How do agents fail on autoresearch: end-to-end diagnostic evaluation on 100 real-world frontier research tasks. arXiv preprint arXiv:2608.14905. Cited by: Appendix D, Appendix H, §4.4.
• Fu et al. (2026) K. Fu, L. Lyu, S. Li, S. Huang, S. Xu, J. Zheng, X. Liu, S. Liu, G. Barbone, Y. Liu, et al. Agentic laboratories of the future: towards world models for scientific discovery. Preprints.org. Cited by: §2.
• Ghafarollahi and Buehler (2025) A. Ghafarollahi and M. J. Buehler SciAgents: automating scientific discovery through bioinspired multi-agent intelligent graph reasoning. Advanced Materials 37 (22), pp. 2413523. Cited by: §2.
• Ghareeb et al. (2026) A. E. Ghareeb, B. Chang, L. Mitchener, A. Yiu, C. J. Szostkiewicz, D. Shved, G. J. Gyimesi, J. M. Laurent, S. M. Wright, M. T. Razzak, et al. A multi-agent system for automating scientific discovery. Nature, pp. 1–3. Cited by: §2.
• Ghoreishi and Allaire (2019) S. F. Ghoreishi and D. Allaire Multi-information source constrained bayesian optimization: sf ghoreishi and d. allaire. Structural and Multidisciplinary Optimization 59 (3), pp. 977–991. Cited by: §2.
• Google (2025) Google Gemini 2.5 flash-lite. Note: https://ai.google.dev/gemini-api/docs/models/gemini-2.5-flash-liteAccessed: 2026-09-23 Cited by: §4.1.
• Gottweis et al. (2025) J. Gottweis, W. Weng, A. Daryin, T. Tu, A. Palepu, P. Sirkovic, A. Myaskovsky, F. Weissenberger, K. Rong, R. Tanno, et al. Towards an ai co-scientist. arXiv preprint arXiv:2502.18864 2. Cited by: §1, §2.
• Gou et al. (2024) Z. Gou, Z. Shao, Y. Gong, Y. Yang, N. Duan, W. Chen, et al. Critic: large language models can self-correct with tool-interactive critiquing. In International Conference on Learning Representations, Vol. 2024, pp. 57734–57811. Cited by: §1, §2.
• Harris et al. (2025) S. B. Harris, R. Vasudevan, and Y. Liu Active oversight and quality control in standard bayesian optimization for autonomous experiments. npj Computational Materials 11 (1), pp. 23. Cited by: §2.
• Houlsby et al. (2011) N. Houlsby, F. Huszár, Z. Ghahramani, and M. Lengyel Bayesian active learning for classification and preference learning. arXiv preprint arXiv:1112.5745. Cited by: §2.
• Huang et al. (2024) J. Huang, X. Chen, S. Mishra, H. S. Zheng, A. Yu, X. Song, and D. Zhou Large language models cannot self-correct reasoning yet. In International conference on learning representations, Vol. 2024, pp. 32808–32824. Cited by: §1, §2.
• Kamoi et al. (2024) R. Kamoi, Y. Zhang, N. Zhang, J. Han, and R. Zhang When can llms actually correct their own mistakes? a critical survey of self-correction of llms. Transactions of the Association for Computational Linguistics 12, pp. 1417–1440. Cited by: §1, §2.
• Kandasamy et al. (2017) K. Kandasamy, G. Dasarathy, J. Schneider, and B. Póczos Multi-fidelity bayesian optimisation with continuous approximations. In International conference on machine learning, pp. 1799–1808. Cited by: §2.
• Kirgis et al. (2026) P. Kirgis, S. Kapoor, A. Schwartz, S. Rabanser, D. Africa, K. Voudouris, V. Nguyen, T. Pilditch, M. Dubois, H. Coppock, et al. Can ai agents conduct open-ended ai research? early evidence from two case studies. arXiv preprint arXiv:2607.27191. Cited by: §1, §2.
• Laurent et al. (2024) J. M. Laurent, J. D. Janizek, M. Ruzo, M. M. Hinks, M. J. Hammerling, S. Narayanan, M. Ponnapati, A. D. White, and S. G. Rodriques Lab-bench: measuring capabilities of language models for biology research. arXiv preprint arXiv:2407.10362. Cited by: §2.
• Liu et al. (2026) J. Liu, J. Pei, J. Huang, C. Si, A. Qu, X. Tang, R. Lu, L. Chen, X. Bai, H. Zheng, et al. The last human-written paper: agent-native research artifacts. arXiv preprint arXiv:2604.24658. Cited by: §2.
• Lu et al. (2024) C. Lu, C. Lu, R. T. Lange, J. Foerster, J. Clune, and D. Ha The ai scientist: towards fully automated open-ended scientific discovery. arXiv preprint arXiv:2408.06292. Cited by: §1, §2, §4.1.
• MacKay (1992) D. J. MacKay Information-based objective functions for active data selection. Neural computation 4 (4), pp. 590–604. Cited by: §2.
• Madaan et al. (2023) A. Madaan, N. Tandon, P. Gupta, S. Hallinan, L. Gao, S. Wiegreffe, U. Alon, N. Dziri, S. Prabhumoye, Y. Yang, et al. Self-refine: iterative refinement with self-feedback. Advances in neural information processing systems 36, pp. 46534–46594. Cited by: §1, §2.
• Mikkola et al. (2023) P. Mikkola, J. Martinelli, L. Filstroff, and S. Kaski Multi-fidelity bayesian optimization with unreliable information sources. In International Conference on Artificial Intelligence and Statistics, pp. 7425–7454. Cited by: §2.
• Mitchener et al. (2025a) L. Mitchener, J. M. Laurent, A. Andonian, B. Tenmann, S. Narayanan, G. P. Wellawatte, A. White, L. Sani, and S. G. Rodriques Bixbench: a comprehensive benchmark for llm-based agents in computational biology. arXiv preprint arXiv:2503.00096. Cited by: §2.
• Mitchener et al. (2025b) L. Mitchener, A. Yiu, B. Chang, M. Bourdenx, T. Nadolski, A. Sulovari, E. C. Landsness, D. L. Barabasi, S. Narayanan, N. Evans, et al. Kosmos: an ai scientist for autonomous discovery. arXiv preprint arXiv:2511.02824. Cited by: §2.
• Narayanan et al. (2024) S. Narayanan, J. D. Braza, R. Griffiths, M. Ponnapati, A. Bou, J. Laurent, O. Kabeli, G. Wellawatte, S. Cox, S. G. Rodriques, et al. Aviary: training language agents on challenging scientific tasks. arXiv preprint arXiv:2412.21154. Cited by: §2.
• OpenAI (2026) OpenAI GPT-5.6. Note: https://openai.com/index/gpt-5-6/Accessed: 2026-09-23 Cited by: §4.1.
• Poloczek et al. (2017) M. Poloczek, J. Wang, and P. Frazier Multi-information source optimization. Advances in neural information processing systems 30. Cited by: §2.
• Pu et al. (2025) Y. Pu, T. Lin, and H. Chen Piflow: principle-aware scientific discovery with multi-agent collaboration. arXiv preprint arXiv:2505.15047. Cited by: §1, §2, §4.1.
• Pu et al. (2026) Y. Pu, T. Lin, and H. Chen Principle-evolvable scientific discovery via uncertainty minimization. arXiv preprint arXiv:2602.06448. Cited by: Table 8, §1, §1, §1, §2, §3.1, §3.3, §3, §4.1, §4.1, §4.2, Table 3.
• Rangelov (2026) B. Rangelov Sibyl: temporal backtesting for literature-based scientific discovery with large language model agents. In ICML 2026 AI for Science Workshop, Cited by: §2.
• Ríos-García et al. (2026) M. Ríos-García, N. Alampara, C. Gupta, I. Mandal, S. Mannan, A. A. Aghajani, N. Krishnan, and K. M. Jablonka AI scientists produce results without reasoning scientifically. arXiv preprint arXiv:2604.18805. Cited by: §1, §2.
• Ruan et al. (2024) Y. Ruan, C. Lu, N. Xu, Y. He, Y. Chen, J. Zhang, J. Xuan, J. Pan, Q. Fang, H. Gao, et al. An automatic end-to-end chemical synthesis development platform powered by large language models. Nature communications 15 (1), pp. 10160. Cited by: §2.
• Schmidgall et al. (2025) S. Schmidgall, Y. Su, Z. Wang, X. Sun, J. Wu, X. Yu, J. Liu, M. Moor, Z. Liu, and E. Barsoum Agent laboratory: using llm agents as research assistants. Findings of the Association for Computational Linguistics: EMNLP 2025, pp. 5977–6043. Cited by: §1, §2.
• Shinn et al. (2023) N. Shinn, F. Cassano, A. Gopinath, K. Narasimhan, and S. Yao Reflexion: language agents with verbal reinforcement learning. Advances in neural information processing systems 36, pp. 8634–8652. Cited by: §1, §2.
• Skarlinski et al. (2024) M. D. Skarlinski, S. Cox, J. M. Laurent, J. D. Braza, M. Hinks, M. J. Hammerling, M. Ponnapati, S. G. Rodriques, and A. D. White Language agents achieve superhuman synthesis of scientific knowledge. arXiv preprint arXiv:2409.13740. Cited by: §2.
• Sui et al. (2015) Y. Sui, A. Gotovos, J. Burdick, and A. Krause Safe exploration for optimization with gaussian processes. In International conference on machine learning, pp. 997–1005. Cited by: §2.
• Swersky et al. (2013) K. Swersky, J. Snoek, and R. Adams Multi-task bayesian optimization. Advances in neural information processing systems 26. Cited by: §2.
• Tang et al. (2026) J. Tang, L. Xia, Z. Li, and C. Huang Ai-researcher: autonomous scientific innovation. Advances in Neural Information Processing Systems 38, pp. 9481–9520. Cited by: Appendix D, §2, §4.1, §4.3.
• Team (2025) Q. Team Qwen3 technical report. External Links: 2505.09388, Link Cited by: §4.1.
• Wang and Buehler (2026) F. Y. Wang and M. J. Buehler Self-revising discovery systems for science: a categorical framework for agentic artificial intelligence. arXiv preprint arXiv:2606.01444. Cited by: §2.
• Yamada et al. (2025) Y. Yamada, R. T. Lange, C. Lu, S. Hu, C. Lu, J. Foerster, J. Clune, and D. Ha The ai scientist-v2: workshop-level automated scientific discovery via agentic tree search. arXiv preprint arXiv:2504.08066. Cited by: §1, §2, §4.1.
• Yang et al. (2025) Z. Yang, W. Liu, B. Gao, T. Xie, Y. Li, W. Ouyang, S. Poria, E. Cambria, and D. Zhou Moose-chem: large language models for rediscovering unseen chemistry scientific hypotheses. In International Conference on Learning Representations, Vol. 2025, pp. 33251–33277. Cited by: §2.
• Yao et al. (2022) S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao React: synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629. Cited by: §2, §4.1.
• Zhang et al. (2026) Y. Zhang, N. Seedat, Y. Dong, P. Cui, J. Zhu, and M. van de Schaar Guideline-grounded evidence accumulation for high-stakes agent verification. arXiv preprint arXiv:2603.02798. Cited by: §2.
• Zhou et al. (2026) Y. Zhou, Y. Wang, J. Wang, H. Zhu, Z. Wang, C. Hiesh, K. Ying, J. Wang, Y. Xu, T. Hou, et al. ADMET-evo: a self-evolving scientific agent for sustained research across heterogeneous tasks. arXiv preprint arXiv:2609.10121. Cited by: §2.
• Zhu et al. (2026) X. Zhu, Y. Cai, Z. Liu, C. Wang, F. Li, W. Jin, W. Liu, Z. Bing, B. Zheng, J. Chai, et al. EvoMaster: a foundational evolving agent framework for agentic science at scale. arXiv preprint arXiv:2604.17406. Cited by: §2.

Appendix A Evidence-Admission Hyperparameters and Sensitivity Analysis

Table 4 summarizes the default hyperparameter values for the Triage  admission gate. All five parameters are fixed across all benchmark tasks and backbones without task-specific tuning.

Figure 6 presents the two-dimensional sensitivity phase map across decision thresholds. The default held-out threshold (τgen=0.16) lies safely within the zero-error region, requiring over ∼80% signal attenuation before a true mechanism is mistakenly rejected.

Appendix B Exploration Diversity (APD) and Exploitation Trajectories (AUOC)

Candidate exploration diversity (Average Pairwise Distance, APD) and cumulative exploitation throughput (Area Under the Optimization Curve, AUOC) across all three backbone LLMs are reported in Table 2 in the main text. Figure 4 plots the task-specific best-so-far convergence trajectories under Qwen3-235B.

Appendix C Execution Latency and Wall-Clock Throughput Profiles

Figure 8 details wall-clock execution latency and scientific discovery throughput across all eight methods under Qwen3-235B. Triage-PiEvo  requires only 134.5​s per run (+13.6% overhead over standard PiEvo ), avoiding the heavy latencies of multi-agent debate or tree search while achieving the highest exploitation throughput (71.73% AUOC, Table 2).

Appendix D Cross-Harness Benchmarks and Triage as a Plug-in Module

We evaluate on the optimization tasks from AutoResearchEval (Fei et al., 2026), a 100-task benchmark spanning seven domains and the research lifecycle. Table 5 benchmarks Triage-PiEvo  against four autonomous agent harnesses: Claude Code, Codex CLI, AI-Researcher (Tang et al., 2026), and ResearchHarness. Among the compared systems, Triage-PiEvo  obtains the highest Official Score (0.418±0.028) and Target Attainment (0.386). While the comparison agents show a gap between raw official score and externally verified score, Triage-PiEvo  attains a Verified Score of 0.382. It also reduces Critical Failures to 5.2% (compared with 14.8% for Claude Code and 26.2% for ResearchHarness), with 86.5% valid runs and a verification cost of $0.92 per task.

Triage as a Plug-in Module.

To test whether the benefits of evidence admission are tied specifically to the PiEvo  candidate generator or transfer across host architectures, Table 6 evaluates Triage  as a verification layer on top of three harnesses (ResearchHarness, AI-Researcher, and Claude Code), comparing Base execution, standard Self-Review prompting, and our + Triage  gate. Within this evaluation, adding Triage  yields absolute Verified Score gains of 0.098 on ResearchHarness, 0.087 on AI-Researcher, and 0.097 on Claude Code. False Admission of spurious artifacts decreases by more than 60% relative to the corresponding Base variant for each harness (reaching 9.6% on Claude Code), while Revision Accuracy ranges from 68.4% to 78.2% across harnesses, with additional token and runtime cost. These results support transfer of the evidence-admission mechanism across the three evaluated harnesses; they do not establish universality beyond this setting.

Appendix E Controlled Stress Tests and Detailed Per-Condition Protocols

This section details the formal experimental protocols, simulation environments, and per-condition parameterizations for the controlled stress evaluations summarized in Section 4.4.

E.1 Protocol for Evaluator Corruptions and Artifact Injections (Table 7, Panel A)

To benchmark the robustness of evidence admission against spurious feedback, we systematically inject non-generalizing anomalies into the observation channel across all four benchmark domains (MBO, NHO, SPO, TMC). A total of 96 controlled evaluation cells are designed across three distinct failure regimes:

• •

  Transient Noise and Execution Dropout (znoise,zexec, 24 cells): For each of the 4 tasks, we inject sporadic stochastic failures under 3 configurations across 2 independent random seeds: (i) zero-mean impulse noise yt←yt+δ with δ∼𝒩⁡(0,5​σt), (ii) heavy-tailed Cauchy perturbations, and (iii) simulated execution timeouts returning default fallback scalars. These anomalies represent transient execution artifacts; they are detected and filtered at Stage 1 (C1: same-channel repeat).

• •

  Channel-Specific Faults (zfault, 24 cells): In multi-channel setups, a single instrument channel may suffer systematic calibration drift. We simulate this across 4 tasks under 2 fault regimes across 3 seeds: (i) an additive offset yt←yt+3​σt, and (ii) a multiplicative scale distortion yt←1.5​yt. These anomalies reproduce consistently under C1, but fail C2 (cross-channel check |vtc−vtc′|≤ϵind), preventing unwarranted principle proposals.

• •

  Hidden Surrogate and Out-of-Distribution Artifacts (zsurr,zood,zdrift, 48 cells): Machine learning surrogate evaluators often exhibit pathological pseudo-optima in out-of-distribution regions. Across 4 tasks, we design 4 boundary failure modes across 3 seeds: (i) high-order polynomial extrapolation explosion, (ii) sparse kernel density estimation under-estimation artifacts, (iii) unregularized feature co-occurrence spikes, and (iv) gradual covariate shift. Because primary and cross-check evaluators share the surrogate training distribution, these artifacts pass C1 and C2, but fail generalization on the disjoint test set during Stage 2b (C3: held-out verification), triggering quarantine.

E.2 Protocol for Planted Scientific Mechanisms (Table 7, Panel B)

To confirm that the admission gate does not filter genuine discoveries, we embed deterministic ground-truth physical mechanisms into the evaluation landscape across 144 independent trials:

• •

  Planted Discontinuous Step Shift (72 trials): We simulate abrupt phase transitions or critical concentration thresholds. For candidate x, the true latent target satisfies y∗(x)=fbase(x)+Δ⋅𝕀[xd>θ], where xd is a target design dimension, θ is a planted threshold, and Δ=2.5​σ0 denotes the effect size. We evaluate across 4 tasks, 3 step thresholds θ, and 6 independent seeds (4×3×6=72 trials).

• •

  Planted Synergistic Interaction Peak (72 trials): We model non-linear molecular binding cooperativity or dual-dopant superconductivity enhancement via a localized interaction term y∗(x)=fbase(x)+γexp(−∥x𝒮−x𝒮∗∥2/2ℓ2), where 𝒮 indexes a coupled feature pair. We evaluate across 4 tasks, 3 feature pairs, and 6 seeds (4×3×6=72 trials).

A mechanism is scored as successfully recalled if an anomalous candidate triggers a principle proposal that passes C1–C3 and reaches the Accepted state within the 24-round budget. Ungated baselines suffer from confirmation masking (where prior admitted artifacts misguide the GP prior and preempt exploration), while Triage-PiEvo  achieves 100.0% recall across all 144 trials.

E.3 Protocol for Proxy Evaluator Exploitation (Table 8)

In automated discovery, agents often optimize against fast proxy evaluators f^proxy​(x) rather than high-fidelity ground truth ftrue​(x). We construct a benchmark environment where f^proxy contains an exploitable shortcut: maximizing a non-physical descriptor dimension xspur inflates f^proxy​(x)>0.90, whereas the ground-truth held-out evaluator ftrue​(x) heavily penalizes xspur due to steric clash or structural instability (ftrue​(x)<0.40). Each agent is allocated T=24 rounds. We measure: (i) the number of exploit shortcut candidates admitted into the search history or persistent principles, (ii) the apparent proxy score reported by the agent, and (iii) the true generalization score on ftrue​(Xheldout).

E.4 Protocol and Case Trace for Dynamic Mechanism Reversal

To assess revision reversibility under non-stationary environments, we introduce a controlled environmental polarity flip at round t=17. Specifically, the optimal objective landscape is inverted (y←ymax−y), rendering all previously learned principles detrimental. We track the chronological recovery trajectory:

• •

  Anomaly Detection (t=17): The inverted feedback induces a residual st>τanom, immediately opening diagnostic state.

• •

  Cascade Retraction (t=18): Held-out test on the candidate yields negative gain Δheldout≤−τgen, moving the invalidated principle to Retracted and recursively retracting its descendants.

• •

  Optimum Recovery (t=23): With the search space unburdened from poisoned prior principles, the AFE probe flow explores the inverted landscape and identifies a candidate reaching ≥95% of the new global optimum at round 23, whereas ungated baselines remain trapped in the invalidated subspace.

Note on Complete Discrimination in Synthetic Stress Tests. The 0.0% false admission rate and 100.0% mechanism recall in Table 7 result from the deterministic logic of the staged gate: transient noise is eliminated at Stage 1 (C1), channel-specific bias cannot satisfy cross-channel invariance at Stage 2a (C2), and surrogate boundary artifacts fail the held-out generalization test at Stage 2b (C3). Conversely, genuine planted mechanisms with significant effect size (Δ≥2.5​σ0) reliably produce positive held-out gains exceeding τgen, allowing full recovery without gate obstruction.

Appendix F Systematic Ablation of Evidence Admission Mechanisms

Table 9 evaluates each component of the Triage  admission policy: the confirmation threshold k∈{1,2,3}, held-out verification, channel reliability tracking, recursive backtracking on retraction, and treating a same-source repeat as independent. Two comparisons carry the argument. Removing held-out verification raises false revision from 4.8% to 34.5%, and treating a same-source repeat as independent raises it to 45.8%—both admit evidence that is reproducible but not transferable, which is the artifact case. Removing channel reliability raises false revision to 19.2% through a different route: verification actions are then routed without regard to which channels have been reconcilable, so corroboration is sought from channels already in disagreement. The gate therefore depends on both the standard of proof and the choice of who supplies it.

Appendix G Algorithmic Specification of the TRIAGE Decision Protocol

Algorithm 1 formalizes the executable decision loop of the Triage  evidence-admission gate.

Appendix H Extended Benchmark Protocols and Failure Metric Definitions

AutoResearchEval Benchmark.

AutoResearchEval (Fei et al., 2026) evaluates multi-step targeted optimization across molecular properties, material bandgaps, and algorithmic code generation. Each harness is run across 5 independent seeds with an identical per-task monetary cap ($2.00) and time budget.

Failure Metric Definitions.

• •

  Official Score: Raw target optimization metric reported by the host framework upon run termination.

• •

  Target Attainment: Binary indicator indicating whether the best discovered candidate achieves ≥90% of the task-specific global target optimum.

• •

  Verified Score: Score evaluated independently by an external, sandboxed ground-truth testbed.

• •

  Valid Runs (%): Percentage of runs that complete all execution rounds without unhandled runtime crashes, tool timeouts, or syntax failure.

• •

  Critical Failure (%): Percentage of runs in which unverified, hallucinated tool outputs or corrupted feedback poisoned the agent’s internal memory, leading to permanent loop collapse or degenerate repeating actions.

注意转载授权 · 阅读原文