RIVO：为 Deep Research 智能体提供轻量自适应重路由

   Jiatan Huang    Hong Yu Affiliation: University of Massachusetts, Amherst Affiliation:  University of Connecticut

Abstract

Deep Research agents rely on long-horizon reasoning and iterative tool use to solve complex information-seeking tasks. However, adaptive decision-making over these trajectories remains challenging, as agents may stop prematurely when further evidence is still needed, continue unnecessarily after sufficient evidence has been gathered, or fail to redirect unproductive trajectories. Existing solutions either require costly model retraining and dense turn-level supervision, or rely solely on heuristic control rules that are not directly optimized for downstream research utility. This motivates a lightweight, utility-aware alternative for deep research control. We formulate adaptive rerouting in long-horizon Deep Research as a state-conditioned trajectory correction problem. Based on this view, we introduce RIVO, a lightweight plug-in framework for adaptive trajectory correction. The agent first estimates its research state from lightweight state signals and determines whether trajectory redirection is needed. Once triggered, the plug-in modifies the Host token logits to steer generation toward a more promising research direction. Theoretical analysis shows that, under a value-alignment condition and bounded local curvature, a sufficiently small logit intervention can locally improve the expected downstream research utility. A curated dataset, BrowseCore, with a paired local corpus is further constructed for efficient and controlled Deep Research experimentation. Building on this resource, the plug-in is trained from downstream outcomes, enabling it to learn corrections that improve subsequent research and final task performance. Extensive experiments across multiple Deep Research benchmarks and control settings demonstrate the effectiveness of our method, showing that lightweight logit intervention can improve long-horizon research without retraining the underlying Host model¹¹ 1 The code and the BrowseCore dataset will be available at: https://github.com/ToneLi/RIVO_deepresearch.

1 Introduction

Large language models (LLMs) have evolved from passive text generators into autonomous agents that interleave reasoning with external tool use, such as web search and knowledge retrieval (Alzubi et al., 2025; Li et al., 2026c; Li et al., 2026b; Li et al., 2026d). Building on this capability, Deep Research agents perform long-horizon, iterative retrieval and reasoning to solve complex information-seeking tasks that require multi-hop evidence acquisition and synthesis(Dong et al., 2026; Li et al., 2026f). These agents are increasingly useful for applications such as web-scale question answering and patient diagnosis and clinical decision support (Wang et al., 2026c). However, current Deep Research systems still suffer from poor search control, as agents may continue searching after sufficient evidence has been collected, stop prematurely when further evidence is still needed, or remain on an unproductive research path instead of redirecting toward more useful evidence (Soudani et al., 2026). This highlights a key limitation of current Deep Research agents. Long-horizon research requires reliable decisions on when to stop, when to reroute, and how to redirect the search toward more promising directions.

A natural way to improve research control is to further train the underlying Deep Research model. Supervised fine-tuning can teach explicit intervention behaviors, but it requires high-quality trajectories that specify not only when the agent should stop or reroute, but also how the research trajectory should be redirected. Collecting such supervision becomes increasingly difficult as research trajectories extend to dozens or even hundreds of interaction steps (Deng et al., 2024b). Reinforcement learning offers a more flexible alternative by optimizing search behavior from task rewards, yet long-horizon optimization introduces sparse and delayed feedback, difficult credit assignment, and substantial training cost due to repeated reasoning and retrieval rollouts (Wang et al., 2026a). These costs become particularly significant when each step invokes large local retrievers or external APIs, and the resulting policy is typically coupled to a specific research backbone, requiring re-optimization when the underlying system changes. Training-free approaches avoid such retraining by relying on prompting or predefined internal and retrieval signals (Yao et al., 2023; Li et al., 2026a). However, these heuristic or proxy signals are not directly optimized for downstream research reward, and therefore cannot explicitly learn the long-term utility of an intervention, whether the agent should stop, whether rerouting is beneficial, and, critically, which alternative search direction should be pursued. These limitations naturally raise the following question: Can we learn a lightweight, plug-and-play research controller that improves decision-making in long-horizon Deep Research without retraining the underlying agent or requiring substantial computational resources?

Motivated by recent advances in inference-time logit reweighting for frozen language models (Hiranandani et al., 2025), we formulate adaptive control in long-horizon Deep Research as a state-conditioned trajectory correction problem. Based on this formulation, we introduce Reward-Learned Logit Intervention for Value Optimization (RIVO), a lightweight plug-in framework for adaptive trajectory correction. Rather than retraining the underlying research model, RIVO estimates the current research state from lightweight state signals to determine whether intervention is needed. When trajectory redirection is triggered, the plug-in applies a token-level correction to the frozen model logits, steering generation toward a more promising research direction. RIVO is trained from downstream trajectory outcomes, allowing it to learn interventions that improve subsequent research and final task performance.

The key component of RIVO is a sparse logit intervention for trajectory correction. A lightweight plug-in generates a low-rank residual correction that is sparsely applied to the Top-K selected logit dimensions, while the Host model, i.e., the underlying Deep Research model, remains frozen. This intervention directly changes selected token preferences and redirects subsequent retrieval without end-to-end retraining. To support model development and controlled evaluation, we construct BrowseCore, a Deep Research dataset with a paired local corpus. Across the evaluated Deep Research benchmarks, RIVO outperforms strong baselines by up to 4.36 points. Further ablation studies validate the key components of RIVO. For example, on BrowseCore, removing the reroute predictor reduces the Success Rate by 5.67 points, while removing the logit intervention leads to a 2.48-point drop. Finally, theoretical analysis studies the effect of logit intervention and shows that sufficiently small, value-aligned interventions can locally improve expected downstream research utility.

• •

  We construct a curated question answer dataset paired with a local corpus to support efficient and controlled training and evaluation of long-horizon Deep Research agents.

• •

  We formulate adaptive rerouting in long-horizon Deep Research as a state-conditioned trajectory correction problem and introduce RIVO, a lightweight plug-in framework for targeted trajectory redirection. Theoretical analysis shows that sufficiently small, value-aligned logit interventions can locally improve expected downstream research utility.

• •

  We conduct extensive experiments across multiple Deep Research benchmarks and control settings. The results demonstrate the effectiveness of RIVO in improving research performance while avoiding end-to-end retraining of the underlying research model.

2 Related Work

Deep Research Agents and Research Control.

Recent Deep Research agents, including LiteResearcher-4B (Li et al., 2026f), Marco-DeepResearch-8B (Zhu et al., 2026), and Tongyi-DeepResearch-30B-A3B (Team, 2025), develop long-horizon research capabilities through supervised and reward-based post-training. Despite strong overall performance, these agents still exhibit limitations in intermediate research control: they may terminate at suboptimal points, continue after meaningful progress has stalled, or fail to redirect unproductive trajectories (Soudani et al., 2026). Recent work has begun to study such decisions. CaRT (Liu et al., 2025) learns when information gathering should terminate from counterfactual trajectories, while LSP (Liu and Wang, 2025) studies adaptive early stopping based on reasoning convergence. However, these methods are not designed for long-horizon Deep Research. Extending such control requires either fine-grained supervision over long trajectories or repeated long-horizon optimization, motivating lightweight control without retraining the full research model.

Plug-in Adaptation for LLM Agents.

Several approaches improve agent behavior through inference-time control. ReAct (Yao et al., 2023) interleaves reasoning and environment actions through prompting, while ASAG (Li et al., 2026a) uses attention states to adaptively terminate unnecessary reasoning. RAAC (Soudani et al., 2026) further introduces retrieval-aware signals to detect stagnation and adapt long-horizon Deep Research trajectories. In contrast, PPDPP (Deng et al., 2024a) learns a lightweight plug-in to guide the policy of a frozen LLM, but targets proactive dialogue rather than long-horizon research. A complementary line of work directly intervenes in model logits: ThinkMerge (Wang et al., 2026b) combines parallel reasoning trajectories through logit averaging, while SWAI (An et al., 2026) steers generation through training-free logit modification. Our method combines these directions by learning a lightweight plug-in that modifies frozen model logits only when trajectory rerouting is needed.

Training Resources for Deep Research.

Existing Deep Research benchmarks provide limited resources for training long-horizon research control. BrowseComp-Plus (Chen et al., 2025) and DRBench (Abaskohi et al., 2026) are primarily designed for evaluation and do not provide dedicated training sets for learning trajectory-level interventions. OpenResearcher (Li et al., 2026g) instead releases 96K synthetically generated long-horizon research trajectories together with an approximately 11B-token retrieval corpus. While this provides a valuable training resource, its large-scale retrieval environment can be computationally expensive for repeated policy optimization, particularly when coupled with a large dense retriever. To support efficient and controlled training of research intervention, we construct a compact training resource that pairs high-quality question–answer examples with a dedicated local corpus, enabling repeated long-horizon rollouts without requiring large-scale retrieval infrastructure.

3 Problem Setup

We consider a long-horizon Deep Research setting in which an LLM agent interacts with a retrieval environment over multiple research turns. Given an initial user question q0, the agent maintains an interaction history ℱt−1 containing previous reasoning, tool calls, retrieved evidence, and tool observations. At each turn t=1,2,…,T, the frozen Host observes q0 and ℱt−1 and produces either a tool action at or a final answer. A tool action is executed by the environment and returns an observation ot=ℰ⁡(at), which is appended to the interaction history. A complete research episode is represented as τ={(a1,o1),(a2,o2),…,(aT,oT)}, followed by a final answer. The environment assigns a task-level reward R⁡(τ) according to the quality of the final research outcome. Among trajectory-level control decisions, in this work, we mainly study adaptive rerouting. At such a state, the agent generates a new retrieval direction qt, which changes the subsequent research trajectory. The quality of a rerouting decision is determined by its downstream effect rather than by the reroute query alone. We quantify the effect of a reroute query qt on subsequent research as

where Qt​(qt) represents the expected research utility after selecting qt at turn t. Our objective is to learn a lightweight trajectory correction mechanism that increases this utility while keeping the host model frozen.

4 RIVO

4.1 The Challenge of Efficient Research Control

Improving long-horizon research control through supervised fine-tuning or reinforcement learning can be computationally expensive. Supervised approaches require fine-grained intervention trajectories, while reward-based optimization depends on repeated long-horizon reasoning and retrieval rollouts with sparse and delayed feedback. Training-free methods avoid model retraining, but typically rely on predefined heuristics or proxy signals that are not directly optimized for downstream research utility. A lightweight plug-in offers a more efficient alternative, but requiring a small model to independently generate complex research directions places substantial generative burden on the controller. We instead retain the frozen Host as the primary generator and use the plug-in to apply residual corrections in logit space. State-guided control determines when intervention is necessary, while the plug-in learns how to modify the Host generation from downstream trajectory-level outcomes.

Motivated by this view, we propose RIVO, a lightweight framework for state-conditioned trajectory correction. As illustrated in Figure 1, RIVO follows a simple pipeline with three components. First, State-Guided Control monitors the evolving research state using lightweight signals and determines whether the current trajectory should stop, continue, or be redirected. Second, when rerouting is triggered, Logit Intervention activates a lightweight plug-in that applies a sparse residual correction to the frozen Host logits, steering generation toward a new retrieval direction without updating the Host parameters. Third, Optimization and Inference trains the plug-in from downstream trajectory-level feedback and selectively applies the learned intervention during inference. Together, these components allow RIVO to adaptively correct unproductive research trajectories while keeping the Host model frozen and limiting intervention to states where rerouting is needed.

4.2 State-Guided Control

Figure 1 summarizes the state-guided control process in RIVO. At each retrieval checkpoint, RIVO evaluates the current research state using three complementary signals: confidence Ct, convergence Dt, and evidence attention At. Confidence identifies states that are sufficiently reliable to stop, convergence determines whether low-confidence states require further verification, and evidence attention distinguishes between continuing the current trajectory and redirecting it. When the state-guided controller selects Continue, a Trained Reroute Predictor performs an additional learned check and may promote the decision to Reroute. Once rerouting is triggered by either component, the plug-in applies token-level logit correction to the frozen Host to redirect subsequent retrieval.

Trained Reroute Predictor.

We collect successful multi-turn Deep Research trajectories and extract a turn-level state st=(q,h<t,at), where q is the original question, h<t is the preceding research history, and at is the current tool action together with its result. A capable language model annotates each state as Continue or Reroute based on the accumulated evidence, current research direction, and gold answer. This process yields 29,252 turn-level examples from 1,475 questions. We split the data by question ID into 26,351 training examples and 2,901 test examples, ensuring that all turns from the same question remain in a single partition. We use the training split to fine-tune Qwen3-0.6B-Base as a lightweight reroute predictor, denoted by P. An action-prediction head produces a distribution pP(⋅∣st) over {Continue,Reroute}, where pP​(Reroute∣st) denotes the predicted rerouting probability. The same plug-in backbone is later used with a separate low-rank residual correction head for logit intervention. Detailed annotation rules, prompts, and data statistics are provided in Appendix E.

Research-state estimation.

At each retrieval checkpoint, RIVO estimates the current research state using three lightweight signals. Provisional-answer confidence Ct measures the Host’s confidence in its current answer using a fixed answer probe. Evidence attention compares the attention assigned to the current and preceding retrieval segments, while attention convergence measures the change in attention entropy relative to the initial retrieval state. We define

Here, Xt denotes the research dialogue at checkpoint t, I is a fixed answer-probe suffix, At denotes the attention assigned to retrieved evidence, and Ht denotes the aggregated attention entropy. Larger Ct indicates higher answer confidence, Δ​At<0 indicates that newly retrieved evidence receives less attention than preceding evidence, and Dt<τH indicates that the attention state has converged. Detailed definitions and computation are provided in Appendix B.2.

State-guided control.

The three signals determine the trajectory-level control action. A high-confidence provisional answer triggers Stop. When confidence remains low but the attention state has converged, RIVO invokes Verify. For a low-confidence and unconverged state, decreasing evidence attention indicates an unproductive retrieval direction and triggers Reroute. Otherwise, the current research trajectory Continues:

For low-confidence states initially assigned Continue, the trained reroute predictor P provides an additional learned check. The decision is promoted to Reroute when y^tP=Reroute. Once Reroute is triggered, the logit-intervention module is activated to redirect subsequent retrieval.

4.3 Logit Intervention

Once Reroute is triggered by the above state-guided control mechanism, RIVO activates the logit-intervention head attached to P. The head is trained from downstream trajectory-level outcomes to learn residual corrections to the Host token logits. During inference, the learned correction is applied while the Host remains frozen, redirecting subsequent retrieval.

Low-rank logit correction.

At reroute step t and token position s, let 𝐳t,sH∈ℝ|𝒱| denote the pre-softmax logits produced by the frozen Host, and let 𝐡t,sP denote the hidden representation produced by the plug-in. We project the plug-in representation through a low-rank correction head,

To keep the intervention sparse, we retain only the K largest-magnitude entries of Δ​𝐳t,s, where t denotes the Deep Research turn at which rerouting is applied, s denotes the token position within the generated reroute query, and j indexes candidate tokens xj∈𝒱 in the vocabulary. Let Mt,s denote the corresponding Top-K mask, The corrected Host logits and resulting token policy are

Here, β controls the intervention strength. In this way, RIVO changes only selected token preferences while leaving the Host parameters unchanged.

Theoretical justification.

We provide a local analysis of how the learned logit intervention affects the expected downstream value of a rerouting decision. Specifically, we characterize the first-order change in trajectory-level reward induced by perturbing the Host logits, and derive a sufficient condition under which the intervention improves the expected downstream value. Let qt=(xt,1,…,xt,Lt) denote the complete reroute query generated when rerouting is triggered at turn t. Here, qt denotes a general textual representation of the rerouting direction; in RIVO, it is instantiated as a structured rerouting hint that is subsequently rewritten by the frozen Host into a search query. By autoregressive factorization,

When β=0, the logit intervention vanishes and the policy reduces to the original Host policy, i.e., π0,θ=πH. For a complete reroute query qt, we define its expected downstream research utility as Qt(qt)=𝔼[R(τ1:T)∣qt,ℱt−1]. The corresponding expected utility under intervention strength β is Jt(β)=𝔼qt∼πβ,θ(⋅∣ℱt−1)[Qt(qt)]. We further define the first-order query-level probability shift as

Then,

Theorem 1 (Local Improvement under Value-Aligned Logit Intervention).

Let

Suppose the intervention direction is locally aligned with downstream value, such that β​Γt>0, and there exists BJ>0 satisfying |Jt′′​(u)|≤BJ for all u between 0 and β. Then

Consequently, for any nonzero β satisfying β​Γt>0, 0<|β|<2​|Γt|BJ, we have Jt​(β)>Jt​(0).

The theorem shows that when the intervention direction is aligned with higher downstream value and its magnitude is sufficiently small, the logit intervention locally improves the expected downstream value relative to the frozen Host policy. Full assumptions, lemmas, and proofs are provided in Appendix I.

4.4 Optimization and Inference

We optimize the lightweight plug-in using trajectory-level feedback while keeping the Host model frozen. For each question q, we sample a group of G complete Deep Research trajectories, 𝒯q={τ1,…,τG}. Whenever Reroute is triggered, the plug-in P applies the logit intervention described in Section 4.3. Let θPlogit denote the trainable parameters of the logit-correction head of P. At token position s, the corrected Host logits are 𝐳t,s=𝐳t,sH+β⁡(Mt,s⊙Δ​𝐳t,s​(θPlogit)), where the Host logits 𝐳t,sH remain fixed. After each trajectory terminates, we use its downstream reward, as described in Appendix C, to compute a group-relative advantage A^i and optimize θPlogit with GRPO:

where ρi,j is the policy ratio and mi,j selects trainable rerouting tokens. Gradients are applied only to θPlogit, while the Host remains frozen. At inference time, the learned correction is activated only when Reroute is triggered. The resulting rerouting hint is rewritten by the frozen Host into a new search query, and the retrieved evidence is fed back into the next research checkpoint.

5 Experiments

Datasets.

We construct BrowseCore, a curated Deep Research dataset paired with a local retrieval corpus for plug-in training and controlled evaluation. BrowseCore contains 2,870 questions for training and development, together with a test set of 359 questions, with each question associated with verified supporting evidence and hard-negative documents. We evaluate RIVO on the 359 question test set. To assess generalization beyond BrowseCore, we additionally evaluate on all 830 BrowseComp-Plus questions (Chen et al., 2025) and the 100-task DRBench benchmark (Abaskohi et al., 2026). Detailed BrowseCore construction procedures are provided in Appendix G.

Baselines and evaluation setup.

We evaluate RIVO on two representative Deep Research systems, LiteResearcher-4B (Li et al., 2026f) and Marco-DeepResearch-8B (Zhu et al., 2026). We compare against four baselines for inference-time trajectory control. ReAct (Yao et al., 2023) interleaves reasoning with tool use through prompting without an explicit trajectory controller. ASAG (Li et al., 2026a) uses attention-state signals to adaptively terminate unnecessary reasoning. RAAC (Soudani et al., 2026) incorporates retrieval-aware signals to detect stagnation and adapt long-horizon research trajectories. PPDPP (Deng et al., 2024a) uses a lightweight plug-in to guide the policy of a frozen language model. For BrowseComp-Plus and BrowseCore, we follow the OpenResearcher evaluation protocol and report Success Rate (SR) using a GPT-4-based judge for final-answer correctness. For DRBench, we use the official score_report implementation with gpt-4o-mini as the judge model, and report Insight Recall (IR), Factuality (Fact.), Report Quality (RQ), Distractor Avoidance (DA), and Harmonic Mean (HM), with HM serving as the overall aggregate metric. All methods use Qwen3-Embedding-8B (Zhang et al., 2025) as the retriever for a controlled comparison.

5.1 Main Results

Table 1 compares RIVO with ReAct (Yao et al., 2023), ASAG (Li et al., 2026a), RAAC (Soudani et al., 2026), and PPDPP (Deng et al., 2024a) across three benchmarks and two Deep Research models. RIVO achieves the strongest overall result across all six model–benchmark settings. For example, with LiteResearcher-4B, it improves over the strongest baseline by 2.41 points on BrowseComp-Plus, 4.15 points on BrowseCore, and 1.45 points in DRBench HM. The gains remain consistent when switching to Marco-DeepResearch-8B, where RIVO also improves over the strongest baseline across all three benchmarks. This cross-model pattern indicates that the benefit of RIVO is not tied to a particular Host architecture, although the magnitude of improvement varies across evaluation settings. We find that Marco-DeepResearch-8B exhibits a different performance profile from LiteResearcher-4B. Despite its larger scale, Marco-DeepResearch-8B performs worse than LiteResearcher-4B on BrowseComp-Plus and BrowseCore, but substantially better on DRBench. This cross-benchmark variation indicates that the two Hosts exhibit different performance profiles, suggesting that benchmark characteristics and model-specific research behaviors may interact in nontrivial ways. On DRBench, RIVO does not achieve the highest score on every individual metric, but it achieves the highest Harmonic Mean for both Host models. With LiteResearcher-4B, RIVO reaches an HM of 34.88, compared with 33.43 for the strongest baseline, while with Marco-DeepResearch-8B, it achieves 44.90 versus 40.54. These results show that RIVO achieves stronger overall performance through more effective trajectory correction.

5.2 Further Analysis

In this section, we systematically analyze the key design choices of RIVO, including the intervention strength β, correction scope K, state-control thresholds λ and τH, the contribution of individual state signals, and the rerouting design. Additional ablations in Appendix A further examine the two functions of the 0.6B plug-in and compare alternative controller configurations, including STOP-only, REROUTE-only, standalone three-way control, and the variant without Verify.

Sensitivity to Intervention Strength and Scope.

There are two important hyperparameters in RIVO, the intervention strength β and the correction scope K. The parameter β controls the magnitude of the residual logit correction applied to the frozen Host, while K determines how many logits are modified during rerouting. Together, they control how strongly and how broadly RIVO redirects the current research trajectory. An intervention that is too weak may fail to change the retrieval direction, whereas an overly strong or overly broad correction may unnecessarily perturb the Host distribution. We therefore study the sensitivity of RIVO to both parameters on the BrowseCore test set using LiteResearcher-4B with a fixed Qwen-Embedding-8B retriever. For questions where rerouting is triggered, we replay the trajectory from the same pre-reroute state while varying β∈{−20,−10,−5,−2,0,2,5,10,20}. The question, trajectory prefix, checkpoint state, retrieval context, and random seed are kept fixed before rerouting, while the reroute query and all subsequent research steps are regenerated. As shown in Figure 2, performance varies substantially with the intervention strength. The best result is obtained at β=−10, achieving a success rate of 24, compared with 16 when β=0. We further vary K∈{8,16,32,64,128,256} to examine the effect of intervention scope. Performance also changes noticeably across different values of K, with the best result achieved at K=128 with a success rate of 24. These results demonstrate that the learned logit intervention in RIVO is effective, with performance improving under appropriately scaled and selective corrections to the Host logits.

Sensitivity to State-Control Thresholds.

We further analyze the two thresholds used by the state-guided controller. The confidence threshold λ determines whether the current trajectory is considered sufficiently reliable, while the convergence threshold τH measures whether the research state has stabilized. We evaluate different combinations of λ∈{0.90,0.95,0.98} and τH∈{−0.05,−0.10,−0.20}. As shown in Figure 3, the best performance is obtained with λ=0.95 and τH=−0.10, achieving an SR of 59.30. A low threshold such as λ=0.90 may over-trust uncertain provisional answers, while a higher threshold requires stronger confidence before termination. The best performance at λ=0.95 suggests that a moderate confidence criterion provides a better balance between premature stopping and unnecessary continued research.

Ablation of State Signals.

We analyze the contribution of the three state signals used by RIVO in Figure 4: confidence Ct, evidence attention At, and convergence Dt, denoted as C, E, and D in the figure, respectively. These signals capture complementary aspects of the current research state: Ct reflects the reliability of the current answer, At measures the attention assigned to retrieved evidence, and Dt indicates whether the research trajectory has converged. Among the individual signals, Evidence Attention At performs best with an SR of 55.03, followed by Confidence Ct at 54.47 and Convergence Dt at 53.20. Pairwise combinations generally improve performance, with {Ct,At} achieving the strongest two-signal result of 56.55 and {Ct,Dt} reaching 56.15. In contrast, {At,Dt} obtains 54.32, suggesting that confidence provides an important complementary signal when combined with either evidence-based or convergence-based information. Using all three signals achieves the best performance of 59.30, outperforming the best single-signal and two-signal settings by 4.27 and 2.75 points, respectively. This shows that the three signals provide complementary information for state-guided trajectory control.

Ablation of Rerouting Design.

The 0.6B plug-in in RIVO has two distinct functions: a reroute predictor, which determines whether a current Continue state should be promoted to Reroute, and a logit-intervention module, which modifies the frozen Host logits after rerouting is triggered to steer the next retrieval direction. In w/o Reroute Predictor, the learned reroute prediction is disabled, so rerouting can only be triggered by the state-guided controller, while the logit intervention is retained. In w/o Logit Intervention, the reroute predictor is retained, but the residual correction to the Host logits is removed, so the Host generates the subsequent rerouting direction without learned logit modification. In w/o Logit Intervention + Reroute Predictor, both learned components are removed, leaving only the state-guided controller for trajectory control. As shown in Table 2, removing the reroute predictor reduces SR from 59.30 to 53.63, while removing the logit intervention reduces SR to 56.82. These results show that both deciding when to introduce an additional reroute and learning how to redirect the frozen Host contribute to the final performance.

6 Conclusion

We present RIVO, a lightweight framework for adaptive trajectory correction in long-horizon Deep Research. RIVO combines state-guided control with a lightweight learned reroute predictor and token-level logit intervention, enabling adaptive decisions about when trajectory redirection is needed and how the frozen Host should be redirected without retraining the Host model. We also construct BrowseCore, a curated question–answer dataset paired with a compact local retrieval corpus for controlled training and evaluation of Deep Research agents. Experiments on BrowseComp-Plus, BrowseCore, and DRBench with two different Deep Research Hosts demonstrate performance gains over existing trajectory-control baselines across all evaluated settings, while ablation studies show that the state signals, reroute prediction, and logit intervention each contribute to the overall performance. Our theoretical analysis further shows that, under a value-alignment condition and bounded local curvature, a sufficiently small logit intervention can locally improve the expected downstream research utility. Overall, these results demonstrate that lightweight, state-conditioned logit intervention provides an effective approach to improving long-horizon Deep Research trajectories.

7 Limitations and Future Directions

The current framework uses relatively constrained rerouting decisions and separates state-based control from the learned logit intervention. Future work could explore more adaptive rerouting strategies that support multiple interventions over a trajectory and jointly learn when to intervene, how strongly to intervene, and which parts of the Host distribution should be modified. Another promising direction is to make the retriever itself adaptive, allowing the retrieval model and trajectory controller to be jointly optimized from downstream feedback so that retrieval representations and rerouting strategies can co-adapt throughout the research process.

References

• Abaskohi et al. (2026) A. Abaskohi, T. Chen, M. Muñoz-Mármol, C. Fox, A. V. Ramesh, É. Marcotte, X. H. Lù, N. Chapados, S. Gella, C. Pal, et al. Drbench: a realistic benchmark for enterprise deep research. In International Conference on Learning Representations, Vol. 2026, pp. 6727–6795. Cited by: §2, §5.
• Alzubi et al. (2025) S. Alzubi, C. Brooks, P. Chiniya, E. Contente, C. von Gerlach, L. Irwin, Y. Jiang, A. Kaz, W. Nguyen, S. Oh, et al. Open deep search: democratizing search with open-source reasoning agents. arXiv preprint arXiv:2503.20201. Cited by: §1.
• An et al. (2026) H. An, S. Park, H. Jin, and Y. Han Steering language models before they speak: logit-level interventions. arXiv preprint arXiv:2601.10960. Cited by: §2.
• Asai et al. (2024) A. Asai, Z. Wu, Y. Wang, A. Sil, and H. Hajishirzi Self-rag: learning to retrieve, generate, and critique through self-reflection. In International conference on learning representations, Vol. 2024, pp. 9112–9141. Cited by: Appendix H.
• Chen et al. (2025) Z. Chen, X. Ma, S. Zhuang, P. Nie, K. Zou, A. Liu, J. Green, K. Patel, R. Meng, M. Su, et al. Browsecomp-plus: a more fair and transparent evaluation benchmark of deep-research agent. arXiv preprint arXiv:2508.06600. Cited by: §2, §5.
• Deng et al. (2024a) Y. Deng, W. Zhang, W. Lam, S. Ng, and T. Chua Plug-and-play policy planner for large language model powered dialogue agents. In International Conference on Learning Representations, Vol. 2024, pp. 10058–10090. Cited by: §2, §5, §5.1.
• Deng et al. (2024b) Y. Deng, W. Zhang, W. Lam, S. Ng, and T. Chua Plug-and-play policy planner for large language model powered dialogue agents. In International Conference on Learning Representations, B. Kim, Y. Yue, S. Chaudhuri, K. Fragkiadaki, M. Khan, and Y. Sun (Eds.), Vol. 2024, pp. 10058–10090. External Links: Link Cited by: §1.
• Dong et al. (2026) Y. Dong, X. Xiao, L. Dong, X. Jin, Z. Li, H. Zhang, D. Wang, and N. Xu S1-deepresearch: beyond search, toward real-world long-horizon research agents. arXiv preprint arXiv:2606.15367. Cited by: §1.
• Fang et al. (2026) J. Fang, Z. Meng, and C. Macdonald SPARKLE: a structured and plug-and-play agentic retrieval policy for adaptive rag models. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 38699–38719. Cited by: Appendix H.
• Guan et al. (2026) X. Guan, J. Zeng, F. Meng, C. Xin, Y. Lu, H. Lin, X. Han, L. Sun, and J. Zhou Deeprag: thinking to retrieve step by step for large language models. In International Conference on Learning Representations, Vol. 2026, pp. 117056–117071. Cited by: Appendix H.
• Hiranandani et al. (2025) G. Hiranandani, H. Wu, S. Mukherjee, and S. Koyejo Logits are all we need to adapt closed models. In Proceedings of the 42nd International Conference on Machine Learning, A. Singh, M. Fazel, D. Hsu, S. Lacoste-Julien, F. Berkenkamp, T. Maharaj, K. Wagstaff, and J. Zhu (Eds.), Proceedings of Machine Learning Research, Vol. 267, pp. 23261–23289. External Links: Link Cited by: §1.
• Huang et al. (2026a) J. Huang, M. Li, Z. Li, S. Kwon, H. Yu, and C. Zhang Counterfactual graph for multi-agent llm calibration. arXiv preprint arXiv:2605.30653. Cited by: Appendix H.
• Huang et al. (2026b) J. Huang, Z. Zhang, T. Ma, M. Li, Y. Zheng, Y. Ye, and C. Zhang GLEN-bench: a graph-language based benchmark for nutritional health. arXiv preprint arXiv:2601.18106. Cited by: Appendix H.
• Huang et al. (2026c) J. Huang, Z. Zhang, K. Shi, Y. Ye, and C. Zhang EvolveRouter: co-evolving routing and prompt for multi-agent question answering. arXiv preprint arXiv:2604.05149. Cited by: Appendix H.
• Kabir et al. (2025) I. Kabir, M. A. Reza, and S. Billah Logic-rag: augmenting large multimodal models with visual-spatial knowledge for road scene understanding. In 2025 IEEE International Conference on Robotics and Automation (ICRA), pp. 10585–10591. Cited by: Appendix H.
• Kim et al. (2026) J. Kim, J. Kim, J. Kim, Y. J. Kim, S. Hwang, and M. Lee Adaptive retrieval for reasoning. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 37366–37381. Cited by: Appendix H.
• Lewis et al. (2020) P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W. Yih, T. Rocktäschel, et al. Retrieval-augmented generation for knowledge-intensive nlp tasks. Vol. 33. Note: Advances in Neural Information Processing Systems Cited by: Appendix H.
• Li et al. (2026a) J. Li, K. Qin, R. Wang, Y. Ma, Q. Chen, M. Li, and S. Liang Stop when further reasoning won’t help: attention-state adaptive generation in reasoning models. In Proceedings of the 43rd International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 306. Cited by: §1, §2, §5, §5.1.
• Li et al. (2026b) M. Li, J. Huang, Z. Yao, and H. Yu Efficient and effective internal memory retrieval for llm-based healthcare prediction. In Findings of the Association for Computational Linguistics: ACL 2026, pp. 35895–35906. Cited by: §1.
• Li et al. (2026c) M. Li, J. Huang, C. Zhang, L. Zhao, and H. Yu In-context optimization for retrieval-augmented generation: a gradient-descent perspective. arXiv preprint arXiv:2605.26356. Cited by: Appendix H, §1.
• Li and Huang (2023) M. Li and L. Huang Understand the dynamic world: an end-to-end knowledge informed framework for open domain entity state tracking. In Proceedings of the 46th international ACM SIGIR conference on research and development in information retrieval, pp. 842–851. Cited by: Appendix H.
• Li et al. (2025a) M. Li, H. Kilicoglu, H. Xu, and R. Zhang Biomedrag: a retrieval augmented large language model for biomedicine. Journal of Biomedical Informatics 162, pp. 104769. Cited by: Appendix H.
• Li et al. (2024a) M. Li, C. Ling, R. Zhang, and L. Zhao Zero-shot link prediction in knowledge graphs with large language models. In 2024 IEEE International Conference on Data Mining (ICDM), pp. 753–760. Cited by: Appendix H.
• Li et al. (2026d) M. Li, H. Zeng, Z. Qian, J. Huang, S. Kwon, H. Zamani, and H. Yu RICE-po: turning retrieval interactions into credit signals for reasoning agents. arXiv preprint arXiv:2605.26352. Cited by: Appendix H, §1.
• Li et al. (2026e) M. Li, Z. Zhan, J. Huang, J. Yeung, K. Ding, A. Blaes, S. Johnson, H. Liu, H. Xu, and R. Zhang CancerLLM: a large language model in cancer domain. NPJ Digital Medicine 9 (1), pp. 266. Cited by: Appendix H.
• Li et al. (2025b) M. Li, Z. Zhan, H. Yang, Y. Xiao, H. Zhou, J. Huang, and R. Zhang Benchmarking retrieval-augmented large language models in biomedical nlp: application, robustness, and self-awareness. Science Advances 11 (47), pp. eadr1443. Cited by: Appendix H.
• Li et al. (2024b) M. Li, H. Zhou, H. Yang, and R. Zhang RT: a retrieving and chain-of-thought framework for few-shot medical named entity recognition. Journal of the American Medical Informatics Association 31 (9), pp. 1929–1938. Cited by: Appendix H.
• Li et al. (2026f) W. Li, B. Qu, B. Pan, J. Zhang, Z. Liu, P. Zhang, W. Chen, and B. Zhang LiteResearcher: a scalable agentic rl training framework for deep research agent. arXiv preprint arXiv:2604.17931. Cited by: §1, §2, §5.
• Li et al. (2026g) Z. Li, D. Jiang, X. Ma, H. Zhang, P. Nie, Y. Zhang, K. Zou, J. Xie, Y. Zhang, and W. Chen Openresearcher: a fully open pipeline for long-horizon deep research trajectory synthesis. arXiv preprint arXiv:2603.20278. Cited by: §2.
• Liu et al. (2025) G. Liu, Y. Qu, J. G. Schneider, A. Singh, and A. Kumar CaRT: teaching LLM agents to know when they know enough. CoRR abs/2510.08517. External Links: Link, Document, 2510.08517 Cited by: §2.
• Liu et al. (2026) W. Liu, X. Ma, Y. Zhu, Y. Li, D. Shi, D. Yin, and Z. Dou Agentic-r: learning to retrieve for agentic search. In Findings of the Association for Computational Linguistics: ACL 2026, pp. 15987–16005. Cited by: Appendix H.
• Liu and Wang (2025) X. Liu and L. Wang Answer convergence as a signal for early stopping in reasoning. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, C. Christodoulopoulos, T. Chakraborty, C. Rose, and V. Peng (Eds.), Suzhou, China, pp. 17896–17907. External Links: Link, Document, ISBN 979-8-89176-332-6 Cited by: §2.
• Penedo et al. (2024) G. Penedo, H. Kydlíček, A. Lozhkov, M. Mitchell, C. Raffel, L. Von Werra, T. Wolf, et al. The fineweb datasets: decanting the web for the finest text data at scale. Advances in Neural Information Processing Systems 37, pp. 30811–30849. Cited by: §G.1.
• Soudani et al. (2026) H. Soudani, E. Lingg, F. Hasibi, and N. Rekabsaz When deep research agents stagnate: enhancing reasoning with retrieval-aware agent control. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, Cited by: §1, §2, §2, §5, §5.1.
• Team (2025) T. D. Team Tongyi deepresearch: a new era of open-source ai researchers. Note: https://github.com/Alibaba-NLP/DeepResearch Cited by: §2.
• Trivedi et al. (2023) H. Trivedi, N. Balasubramanian, T. Khot, and A. Sabharwal Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions. In Proceedings of the 61st annual meeting of the association for computational linguistics (volume 1: long papers), pp. 10014–10037. Cited by: Appendix H.
• Wang et al. (2026a) G. Wang, S. Dai, G. Ye, Z. Gan, W. Yao, Y. Deng, X. Wu, and Z. Ying Information gain-based policy optimization: a simple and effective approach for multi-turn search agents. In The Fourteenth International Conference on Learning Representations, External Links: Link Cited by: §1.
• Wang et al. (2026b) H. Wang, C. Du, K. Kawaguchi, and T. Pang Think in parallel, answer as one: logit averaging for open-ended reasoning. In International Conference on Learning Representations, Vol. 2026, pp. 77445–77462. Cited by: §2.
• Wang et al. (2026c) Z. Wang, C. Wei, J. Chan, R. Leaman, C. Day, C. Wu, M. A. Knepper, A. S. Farias, J. Rincon-Torroella, H. Slika, et al. DeepER-med: advancing deep evidence-based research in medicine through agentic ai. arXiv preprint arXiv:2604.15456. Cited by: §1.
• Yao et al. (2023) S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao ReAct: synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), Cited by: Appendix H, §1, §2, §5, §5.1.
• Zhang et al. (2025) Y. Zhang, M. Li, D. Long, X. Zhang, H. Lin, B. Yang, P. Xie, A. Yang, D. Liu, J. Lin, et al. Qwen3 embedding: advancing text embedding and reranking through foundation models. arXiv preprint arXiv:2506.05176. Cited by: §G.1, §5.
• Zhu et al. (2026) B. Zhu, Q. Jia, T. Lan, J. Ren, F. Gu, F. Jiang, L. Wang, Z. Xu, and W. Luo Marco deepresearch: unlocking efficient deep research agents via verification-centric design. arXiv preprint arXiv:2603.28376. Cited by: §2, §5.

Appendix Navigation

B.1 Structural Ablations .B.1

F  Prompt .F

G  BrowseCore .G

G.2.2 Candidate Document Sources .G.2.2

G.2.5 Hard-Negative Mining .G.2.5

G.2.6 Final Corpus Assembly .G.2.6

Appendix A Ablation study

Ablation of the 0.6B Plug-in.

The 0.6B plug-in in RIVO is used in two distinct stages: first, to predict whether a current Continue state should be promoted to Reroute, and second, to modify the frozen Host logits after rerouting is triggered. We separately remove each function while keeping the remaining pipeline unchanged. Removing the 0.6B reroute predictor reduces SR from 59.30 to 53.63, indicating that the learned reroute decision contributes substantially to identifying useful intervention points. When the reroute decision is retained but the 0.6B logit intervention is removed, the system directly outputs the rerouting hint without modifying the Host logits and achieves an SR of 56.82. The degradation in both variants shows that the two uses of the 0.6B plug-in are complementary: the predictor helps determine when rerouting should occur, while the logit intervention controls how the frozen Host is redirected toward a more effective retrieval trajectory.

Controller Design Ablation.

We further examine how different controller designs affect long-horizon research performance. Table 3 compares variants that restrict the lightweight controller to a single action, allow it to independently perform full three-way control, or combine it with the original control mechanism. The definitions of STOP-only, REROUTE-only, and the standalone 0.6B three-way controller are provided in Section B.1. w/o Verify removes the Verify action from the state-guided controller.

Appendix B RIVO’s adaptive rerouting pipeline

B.1 Structural Ablations.

Figure 5 summarizes the control flow of RIVO and the structural variants used in our ablation study. In the full model (1), RIVO first evaluates the current research state using three complementary signals: confidence Ct, convergence Dt, and evidence attention At. Confidence determines whether the current trajectory is sufficiently reliable to terminate. For low-confidence states, convergence identifies whether the current provisional answer should be further verified, while evidence attention compares the usefulness of newly retrieved evidence with previously collected evidence to determine whether the trajectory should be rerouted or continued. If the state-guided controller selects Continue, a lightweight 0.6B reroute predictor performs an additional learned check and may promote the decision to Reroute. Once Reroute is triggered, either by the state-guided controller or the reroute predictor, the 0.6B plug-in applies token-level logit correction to the frozen Host, guiding it toward a new retrieval direction. A Continue decision instead leaves the Host trajectory unchanged and proceeds to the next research step.

We construct four structural variants to isolate different parts of this control mechanism. Variant (2), STOP-only, retains only the confidence-based stopping mechanism. Variant (3), REROUTE-only, retains the rule-based rerouting mechanism while removing the other control branches. Variant (4), Predictor-only, replaces the state-guided controller with the lightweight 0.6B reroute predictor. Variant (5), w/o Verify, removes the verification branch while preserving the remaining control logic. Importantly, these variants modify only how the control decision is made. After a Reroute or Continue decision is produced, variants (2)–(5) follow exactly the same downstream execution as the full model: Reroute invokes the plug-in-based logit correction followed by a new retrieval step, whereas Continue proceeds with the normal Host research trajectory.

When Verify is selected, the current provisional answer is explicitly treated as an unverified hypothesis. The Host receives the following instruction:

The current provisional answer is: {answer}. Treat it as an unverified hypothesis. Check it against the retrieved primary evidence, look for contradictions, and use browser.open/browser.find (or a new search when necessary) to resolve any missing fact before answering.

A Stop decision terminates the research process and requires the Host to return the final response in the format <answer>\boxed{answer}</answer>. We additionally impose a maximum research-turn budget. If this budget is exhausted before a normal stopping decision is reached, further tool calls are disabled and the Host is instructed to synthesize the best-supported answer from the evidence already collected:

The research budget is exhausted. Do not call any tool. Synthesize the best-supported final answer from the evidence already in this conversation. Return only <answer>...</answer>.

B.2 State-guided control

Provisional-answer confidence.

At checkpoint t, let Xt denote the serialized dialogue. We append a fixed answer-probe suffix I directly at the token level and greedily decode a provisional answer Yt=(y1,…,ym). From the selected-token log-probabilities ℓi=log⁡pH​(yi∣Xt,I,y<i), we compute

where m ends at the first complete boxed answer and λ=0.95. Ct provides a lightweight internal estimate of the Host’s confidence in its current provisional answer given the evidence available at the current retrieval state.

Evidence-attention score.

We monitor the final L=4 Host layers over a W=32 token window. Let ah,q,k(l) denote the attention weight from query position q to token k at head h of layer l. Since a retrieval step may return multiple pieces of evidence, we define St as the union of all evidence-token spans accumulated within the current retrieval segment before checkpoint t. For an evidence segment S, we compute

where Nh is the number of attention heads, Q is the number of query positions in the monitored window, and |S| is the number of evidence tokens. Thus, At​(S) measures the average attention assigned by the Host to the evidence accumulated in a retrieval segment. We denote the scores of the previous and current retrieval segments as Atprev=At​(St−1) and Atcur=At​(St), respectively. Their difference, Δ​At=Atcur−Atprev, indicates whether the newly accumulated evidence receives more or less attention than the evidence from the preceding retrieval segment.

Attention convergence.

We use attention entropy to measure whether the Host has reached a more focused and stable research state. For query position q, head h, and layer l, we compute the normalized attention entropy

where lower entropy indicates that attention is concentrated on a smaller set of tokens. We aggregate the entropy across the monitored layers, heads, and query positions as

where H1 is the entropy at the first retrieval checkpoint. Thus, Dt measures how much the attention distribution has become more concentrated relative to the initial research state. We treat the state as converged when Dt<τH, with τH=−0.10.

At each retrieval checkpoint, rerouting can be triggered by either the inference-time controller or the learned 0.6B plug-in. The controller triggers Reroute when the current provisional answer has low confidence, the attention state has not converged, and the newly retrieved evidence receives less attention than the preceding evidence. If the controller instead returns a low-confidence Continue, we consult the plug-in and promote the decision to Reroute when its rerouting probability exceeds a threshold.The final control decision combines the state-guided controller and the learned plug-in trigger:

Here, Δ​At=Atcur−Atprev, and Gt indicates whether the minimum-round and cooldown constraints are satisfied. When gt=1, the plug-in correction head is activated to redirect the subsequent retrieval.

Appendix C Rerouting Reward Details

We use a trajectory-level reward that combines final task success with the quality of the rerouting intervention. For trajectory τi, the reward is

where Rioutcome measures final-answer correctness, Qi measures rerouting-hint quality, and Pi penalizes invalid rerouting outputs.

Outcome reward.

The outcome reward is determined by the final answer of the complete Deep Research trajectory:

Rerouting-hint quality.

Each rerouting hint consists of three fields: Entity, Relation, and Disambiguator. We evaluate the generated hint according to seven criteria:

1. 1.

   Format validity: the output follows the required three-field format;

2. 2.

   English usage: the generated fields are written in English;

3. 3.

   No self-talk: the output does not contain reasoning traces, instructions, or meta-language;

4. 4.

   Entity anchoring: the Entity field is grounded in the question or the observed research history;

5. 5.

   Relation validity: the Relation field provides a meaningful relation for redirecting retrieval;

6. 6.

   Disambiguation validity: the Disambiguator provides additional information that narrows the search direction;

7. 7.

   Field distinctness: the three fields provide non-redundant information.

Let ci,k∈{0,1} indicate whether the k-th criterion is satisfied. We compute

where wk denotes the fixed importance weight of criterion k. We use uniform weights wk=1/7 for all seven criteria.

The rerouting-quality reward is

Thus, valid hints receive a positive reward, while invalid hints receive a graded negative reward according to the satisfied quality criteria.

Invalid-hint penalty.

We additionally define

No invalid-hint penalty is applied when no rerouting hint is generated.

For the configuration used in our main experiments, we set λ=2.0,γ=2.0.

When no rerouting hint is generated, Qi=Pi=0, and the reward reduces to Ri=Rioutcome

Appendix D implementation details

For action prediction, we fine-tuned Qwen3-0.6B-Base using LoRA-based supervised fine-tuning (SFT) with two action labels: Continue, and Reroute. We set the maximum sequence length to 8,192 tokens and trained for 3 epochs using AdamW with a learning rate of 2×10−4, a cosine learning-rate schedule, and a warmup ratio of 0.03. For LoRA, we used a rank of 16, a scaling factor of 32, and a dropout rate of 0.05. Training was conducted on 4 GPUs with a per-device batch size of 1 and 4 gradient accumulation steps, resulting in an effective batch size of 16. To alleviate class imbalance among the action labels, we applied class-aware sampling with an exponent of 0.5.

We trained a Qwen3-0.6B control-and-correction plugin while keeping the Deep Research Host frozen. The plugin was initialized from a two-class SFT controller and adapted using LoRA with rank 16, scaling 32, and dropout 0.05. We used GRPO with five trajectories per question, a training batch size of 8, and a clipping ratio of 0.2. The correction head was optimized using AdamW with a learning rate of 10−4, zero weight decay, and a maximum gradient norm of 1.0. The SFT-initialized controller parameters were kept fixed during GRPO training. No KL or entropy regularization was applied. The structured route hint used token budgets of 16, 12, and 12 for the ENTITY, RELATION, and DISAMBIGUATOR fields, with temperature 1.0, top-p=0.95, and correction Top-K=128. We trained on 2,870 questions, used 359 questions for testing.

We used Qwen/Qwen3-Embedding-8B as the fixed dense retriever. The maximum research horizon was set to 80 rounds, with an input budget of 23,000 tokens and a maximum model context length of 32,768 tokens. For state-based action decisions, we used the host model itself to extract attention-based signals and confidence scores. We set the confidence threshold to 0.95 and the entropy-delta threshold to −0.10, using a 32-token decoding attention window and monitoring the final four layers. It generated an ENTITY ; RELATION ; DISAMBIGUATOR route hint with token budgets of 16, 12, and 12, respectively, using correction Top-K=128. The frozen Host rewrote this hint into a browser.search query using temperature 0.7, a maximum of 32 generated tokens, and a 16-word query limit.

Appendix E Turn-Level Supervision and Data Construction

Turn-Level Supervision.

We collect multi-turn deep research trajectories from a pretrained deep-research agent and retain trajectories with correct final answers. Upon completion of each deep-research turn t, we extract a state st=(q,h<t,at), where q is the original question, h<t is the deep research history preceding the current turn, and at is the current action together with its result. We extract at from the final tool call of turn t, such as search, open, or find. A capable language model, such as GPT-4.1, evaluates the current action at in the context of the question q and the accumulated deep-research history h<t, using the gold answer as the annotation target. It first determines whether the cumulative evidence after at supports all essential parts of the gold answer. If so, it sets ztstop=1. Otherwise, it sets ztstop=0 and evaluates whether the search direction represented by at remains productive given the preceding history. It sets ztreroute=1 if the direction is unproductive and requires a substantial change, and ztreroute=0 if the direction remains productive and should be continued. A single failed tool call is insufficient to trigger rerouting. When ztstop=1, we set ztreroute=0. The resulting label is:

Appendix F reports the trajectory-generation model and annotation prompt. Using these annotations, we initialize the plug-in from Qwen3-0.6B-Base and fine-tune it on the turn-level action labels. Given the current state st=(q,h<t,at), the plug-in generates exactly one label from {Continue,Reroute}. Training details are provided in Appendix D.

Training Data Construction.

We use Tongyi-DeepResearch-30B-A3B to generate complete multi-turn deep-research trajectories for 3,229 questions. To avoid test leakage, trajectories corresponding to the 359 BrowseCore test questions are excluded from controller training. We then use GPT-4.1 to evaluate the correctness of each final answer and retain 1,575 correct trajectories, comprising 32,351 turns ending with actions such as search, open, or find. For each turn t, GPT-4.1 receives the original question, the gold answer, the accumulated deep-research trajectory through the current action result, and the current action type. The gold answer is used only as the annotation target and is not treated as evidence. The judge first determines whether the accumulated evidence is sufficient to answer the question and then, when the evidence is insufficient, determines whether the current search direction remains productive. After annotation-quality filtering, 1,475 questions have complete turn-level annotations, yielding 29,252 examples. We split the data by question ID into 26,351 training examples and 2,901 test examples, ensuring that all turns associated with the same question remain in a single partition. The complete annotation prompt is provided at Figure 8. Turn-level action annotation with GPT-4.1 cost approximately $1,800.

Appendix F Prompt

F.1 Deep Research System Prompt

Figure 6 presents an example system prompt used by LiteResearcher-4B in our experiments. The prompt specifies the agent role, tool-use protocol, reasoning format, and final answer format. It illustrates how the Deep Research agent is instructed to iteratively reason, interact with browser tools, and return a final answer after sufficient information has been collected.

注意转载授权 · 阅读原文