LifeSearch：面向真实生活与生产力场景的 Deep Research 智能体基准

   Senjie Jin    Yansong Ning    Xinbing Liang    Haotian Luo Affiliation: Jiazheng Zhang, Yuxuan HE, Will WU, Clarenceai Affiliation: LLM Department, Tencent

Abstract

Deep research (DR) agents address open-ended information needs through multi-step reasoning, long-horizon web search, and evidence synthesis. However, existing benchmarks primarily focus on academic research or synthetic tasks, leaving DR agents’ capabilities in real-world life and professional productivity settings insufficiently evaluated. To bridge this gap, we introduce LifeSearch, a challenging benchmark comprising 203 Chinese and English tasks derived from real-user query seeds across 7 concrete life and productivity domains. Specially, we spend totally more than 4,500 hours of human annotation to construct over 4,000 expert-written, fine-grained rubrics, covering both report-content criteria for coverage and accuracy, as well as concrete criteria specifying values, units, dates, and URLs. We conduct extensive experiments on 12 advanced DR agents under a unified evaluation harness, systematically analyzing their performance patterns and failure modes. We find that even the best-performing frontier agent, GPT-5.6 Sol, achieves a score of only 63.4%, revealing the potential improvement on realistic life and professional DR tasks. We compare candidate LLM judges against human rubric labels to select the grading configuration. A human ranking study shows strong alignment between aggregate benchmark scores and human judgments, with Spearman’s ρ=0.89 and Kendall’s τ=0.81. LifeSearch provides a rigorous evaluation framework and foundation for advancing DR agents toward real-world life and productivity.

⁰⁰footnotetext: Correspondence to: yibowang0021@gmail.com

1 Introduction

Deep Research (DR) agents solve open-ended information-seeking problems by iteratively interacting with search engines and web pages, planning multi-step retrieval, and summarizing the collected evidence into a comprehensive report (Zhang et al., 2025; Gemini, 2025; OpenAI, 2025). Building on this agentic search paradigm, DR agents demonstrate great potential across knowledge-intensive domains such as software engineering, biomedicine, and finance (Singh et al., 2025; Chen et al., 2025; Zhu et al., 2025). Due to their broad applicability, DR agents attract growing interest from both academia and industry, with increasing efforts devoted to evaluating DR agents.

Many studies evaluate the capability of DR agents. Early benchmarks primarily rely on tasks with easily verifiable, short-form answers, e.g., BrowseComp (Wei and others, 2025) poses fact-seeking questions that can be graded by exact string match against a single gold answer. To better reflect real-world use, subsequent work introduces open-ended, long-horizon tasks and detailed rubrics for evaluating the resulting reports. For example, DeepResearch Bench (Du et al., 2026) and ResearchRubrics (Sharma et al., 2026) score reports against structured, multi-criterion rubrics, DRACO (Zhong et al., 2026) targets cross-domain accuracy, completeness, and objectivity, and Mind2Web 2 (Gou et al., 2025) introduces an agent-as-a-judge framework built on tree-structured rubrics. DeepResearchEval (Wang et al., 2026b) introduces an agentic evaluation pipeline that combines adaptive quality assessment with active fact-checking via web search, while WideSearch (Wong et al., 2026) evaluates broad information collection through structured tables. DR agents are also becoming increasingly capable of addressing practical tasks in everyday life and professional work. Nevertheless, as summarized in Table 1, existing benchmarks provide limited coverage of real-world life and professional productivity settings, with many evaluations remaining centered on academic research tasks or synthetic tasks.

Motivated by this gap, we present LifeSearch, a benchmark for evaluating DR agents on real-world life and professional productivity search. We hired crowd workers to write the initial query seeds, which undergo filtering and expert-guided refinement to form the benchmark tasks. LifeSearch spans 7 concrete life and productivity domains, and its construction involves more than 4,500 hours of human labor to produce over 4,000 expert-written, fine-grained rubrics that comprehensively assess both the coverage and the accuracy of a candidate report. Specifically, LifeSearch comprises 203 tasks: 102 in Chinese and 101 in English. A substantial fraction of our rubrics are numeric and grounded in concrete values, units, or thresholds. Satisfying these criteria therefore requires retrieving and reporting the precise quantities rather than producing qualitatively plausible statements. Furthermore, we compare 11 LLM-judge configurations on 402 human-annotated rubric labels and select Gemini 3.1 Pro with low reasoning effort and zero temperature for benchmark grading. A human ranking study shows that the overall rankings produced by our evaluation closely agree with human rankings, yielding Spearman’s ρ=0.89 and Kendall’s τ=0.81.

We evaluate 12 frontier DR agents under a shared harness (Figure 1), yielding three main findings. i) Substantial headroom remains: Even top-performing GPT-5.6 Sol reaches only 63.4%, with all systems falling below 70% across domains. ii) Quality transcends volume: Higher scores correlate with longer reports, yet volume alone is non-deterministic: Claude Opus 5 and GLM-5.3 have comparable token counts but differ by 6.9 percentage points, highlighting the centrality of evidence synthesis. iii) Common failure modes: Autonomous runs can fail through premature clarification, use of evidence outside the requested time window, and omissions of specific task requirements.

In summary, our contributions are:

• •

  The first benchmark centered on real-world life and professional productivity DR evaluation. LifeSearch shifts the evaluation lens of DR agents toward realistic life and work search, and reveals a concrete capability bottleneck of current frontier models in this setting.

• •

  LifeSearch: a large-scale, expert-annotated bilingual benchmark. We construct and open-source LifeSearch, spanning 7 life and productivity domains and comprising more than 4,000 expert-written, fine-grained rubrics produced through over 4,500 hours of human annotation labor, to comprehensively evaluate both the coverage and the accuracy of DR agent reports.

• •

  A comprehensive empirical study. We carefully examine agreement with human judgments at both the rubric and overall-ranking levels, systematically benchmark 12 current DR agents under a unified harness, providing actionable insight for the DR research community.

2 Related Work

Deep Research Agent. Deep Research (DR) agents extend earlier web-browsing assistants into autonomous systems that interleave multi-step reasoning with iterative search and synthesize the retrieved evidence into a research report, e.g., GAIA (Mialon et al., 2023), WebArena (Zhou et al., 2024), and AssistantBench (Yoran et al., 2024). Early work such as Search-R1 (Jin et al., 2025) and Kimi-Researcher (Moonshot AI, 2025) trains LLMs to interleave reasoning with search-engine calls, enabling emerging agentic research capabilities. Building on this, subsequent studies such as Tongyi DeepResearch (Team et al., 2025), together with harder benchmarks such as BrowseComp (Wei and others, 2025), push DR agents toward long-horizon and more difficult information-seeking behavior. DR agents show strong potential across knowledge-intensive domains, including biomedicine and finance (Chen et al., 2025; Zhu et al., 2025).

Deep Research Agent Evaluation. As DR agents increasingly produce open-ended, long-form reports, evaluating their outputs becomes substantially more challenging than verifying short, exact answers. Subsequent work further decomposes each task into a structured set of criteria: DeepResearch Bench (Du et al., 2026) and ResearchRubrics (Sharma et al., 2026) grade reports against multi-criterion rubrics, ReportBench (Li et al., 2025) evaluates academic survey reports, DRACO (Zhong et al., 2026) targets cross-domain accuracy, completeness, and objectivity, and Mind2Web 2 (Gou et al., 2025) introduces an agent-as-a-judge framework over tree-structured rubrics. More recent work further automates this pipeline: DeepResearchEval (Wang et al., 2026b) combines adaptive quality assessment with active fact-checking, while LiveResearchBench (Wang et al., 2026a) and WideSearch (Wong et al., 2026) extend rubric-style grading to dynamic, user-centric reports and structured-table deliverables, respectively. However, most of these focus on academic research or synthetic tasks, leaving DR agents’ capabilities in real-world life and professional productivity settings underexplored. Their rubrics also lack the numerical grounding, and independent verifiability needed to assess whether a report is truly usable as a real-world life or work deliverable.

3 Overview of LifeSearch

Figure 2 outlines the construction and evaluation pipeline, while Figure 3 summarizes the task distribution and rubric statistics. LifeSearch comprises 203 tasks (102 in Chinese and 101 in English) across seven real-world personal and professional productivity domains. The tasks are developed from query seeds written by hired crowd workers and refined by experts to express complex, pragmatic information-seeking needs. To rigorously evaluate generated reports, trained expert annotators develop 4,014 fine-grained rubrics assessing both requirement coverage and factual accuracy. Crucially, many rubrics are anchored to concrete numerical values, units, dates, or thresholds, facilitating verifiable, criterion-level evaluation.

3.1 Data Collection and Task Domains

To target real-world life and professional productivity scenarios, LifeSearch requires both authentic task provenance and broad coverage of user needs in its construction. We begin with requests written by hired crowd workers. These requests span diverse workflows—including file operations, software engineering, and deep research—providing a heterogeneous pool of realistic user needs.

We construct the data collection pipeline through the following stages:

Stage 1: Intent and quality filtering. We use an LLM to assess search relevance, query quality, and dependence on implicit recent events. We filter out requests unrelated to information seeking, queries with unclear intent or insufficient context, and requests that presuppose an unspecified recent event or state change. General time-sensitive searches remain eligible. Appendix B.1 provides the prompt.

Stage 2: Domain discovery. We use an LLM to classify the retained queries by their core purpose into seven productivity domains: company and competitor research, academic and professional literature, financial market analysis, policy and government data, industry news and trend aggregation, technical tools and developer resources, and everyday facts. Appendix B.2 provides the template.

Stage 3: Task rewriting. Candidate queries may combine information seeking with operational actions such as generating local files or interactive HTML visualizations. We use an LLM to remove these operations and local-file dependencies with minimal changes, preserving the research intent, source constraints, temporal expressions, and content-organization requirements. Appendix B.3 provides the rewriting template.

Stage 4: Manual selection. Experienced AI researchers perform a final screening to retain tasks that are sufficiently complex, diverse, clear, and feasible to evaluate.

Figure 3 summarizes the distribution of the retained tasks across the seven domains. The resulting mix spans both specialized research for professional decision-making and practical information needs arising in everyday life, rather than concentrating on a single subject area.

3.2 Expert-Guided Task Refinement

We invite experienced annotation experts to refine the query seeds retained in Section 3.1 into self-contained research tasks while preserving the original query intent. The same experts also participate in rubric annotation (Section 3.3). The refinement process comprises three stages.

Stage 1: Subtask decomposition. Each seed is decomposed into a small set of complementary subtasks, typically two to four, each with a concise description and one of four types: Lookup for retrieving entities, facts, or values; Compute for deriving quantitative indicators; Compare for examining differences across entities or time periods; and Reason for drawing conclusions and assessing implications. This decomposition makes the information needs explicit and helps identify missing or redundant dimensions.

Stage 2: Capability-oriented enrichment. Experts refine the subtasks around a small set of relevant capabilities, typically two chosen from Search Planning (SP), Information Synthesis (IS), Conflict Resolution (CR), and Uncertainty Handling (UH). SP emphasizes comprehensive coverage of parallel aspects; IS requires integrating evidence from multiple sources; CR introduces source comparison and authority constraints; and UH requires acknowledging missing evidence, providing justified estimates, or stating limitations. Redundant subtasks are merged or removed, and additional requirements are introduced only when needed to probe the selected capabilities. The task thus gains depth while remaining focused on the user’s original goal.

Stage 3: Natural-language rewriting. Finally, experts consolidate the refined subtasks into a coherent query grounded in a concrete life or work scenario. The wording preserves the user’s perspective and naturally incorporates the relevant entities, source requirements, and temporal scope where needed. Final revisions improve clarity, remove repetitive or formulaic phrasing, and ensure that the intended capability requirements remain explicit in the resulting request.

3.3 Rubric Design

For each refined task q, the experts involved in task refinement construct a rubric set Rq describing the requirements of a satisfactory report. Finance experts also participate in annotating rubrics for financial tasks. We use rubric to denote a single scoring requirement that supports an independent Pass/Fail judgment. The benchmark contains 4,014 rubrics, averaging 19.8 per task. Table 2 summarizes the assessment dimensions and annotation fields; detailed writing requirements and illustrative examples are provided in Appendix A. Appendix E presents Chinese and English tasks with selected rubrics and their annotation attributes.

Assessment dimensions. Rubric construction covers seven complementary dimensions: instruction following, information coverage, factual accuracy, evidence and citation quality, reasoning and analysis, task success, and presentation.

Rubric representation. Each rubric records its text (stored as criterion), the acceptance statement; type, distinguishing Objective from Subjective judgments; necessity, distinguishing Explicit requests from Implicit but reasonable requirements; priority, retaining the qualitative Must Have / Nice to Have label supplied to the judge; and weight, an annotator-assigned measure of the rubric’s relative importance within the task. Numerical aggregation uses weight, rather than deriving weights from the priority label. The guidelines prioritize objective rubrics and reserve unavoidable subjective preferences for optional aspects.

Weight assignment. Annotators assign each rubric a weight according to its relative importance for fulfilling the user’s request. We record weights as floating-point values in [0,1] with one decimal place, yielding increments of 0.1. Higher weights indicate more important requirements. These numerical weights are used directly for score aggregation, as detailed in Section 3.4.

Writing and grounding requirements. Rubrics are self-contained, non-redundant, and operationally precise. Experts define target entities, factual targets, units, temporal bounds, and tolerances so that judgments require no additional web browsing. Expert annotators specify and verify factual requirements using authoritative sources. Approximately 40% of the rubrics explicitly specify numerical values, specific events, or named documents, which provide concrete targets for verification.

For open-ended tasks such as recommendations, rubrics evaluate comparative depth and constraint satisfaction rather than a single rigid response, thereby permitting valid alternatives. Rubrics tied to specific examples remain optional unless the prompt explicitly requires them.

Quality control. We implement annotation quality control across two stages: (i) an iterative cycle of annotation, inspection, and revision to refine the rubrics; (ii) outlier screening, where low-scoring tasks are checked for flaws, while overly easy tasks are tightened.

3.4 Evaluation Methodology

We adopt a rubric-based LLM-as-a-Judge approach to assess each final report. The resulting rubric-level judgments are aggregated into a score.

Given a task, its rubric set, and a candidate report, an LLM judge assigns a binary Pass/Fail label to each rubric. Judgments follow the rubric’s acceptance conditions, including factual targets, source requirements, and temporal scope. Semantically equivalent wording and numerical approximations are accepted where permitted by the rubric. Let Rq denote the rubric set for task q, and let yq,r∈{0,1} indicate whether the candidate report for task q satisfies rubric r.

For each rubric r in task q, we use its annotator-assigned weight wq,r, which reflects relative importance within the task. We first normalize the weighted satisfaction score within each task and then average these scores uniformly across the task set Q:

The denominator sums the weights of all rubrics in the task (including unmet ones), requiring a positive total weight for well-defined normalization; zero-weight rubrics are omitted. Averaging sq uniformly across tasks assigns equal importance to each task regardless of rubric count or scale, measuring weighted requirement satisfaction under annotated importance weights.

3.5 Judge Model Selection in DR Setting

For judge selection in the DR rubric setting, we annotate one anonymized model response for each of 20 tasks (10 Chinese, 10 English), covering 402 rubrics. For each report, two to three domain experts independently assign binary Pass/Fail labels, reaching a substantial inter-annotator agreement (mean pairwise Cohen’s κ=0.715). All 65 initial disagreements are subsequently resolved through discussion to form the reference labels used to compare judge configurations.

Benchmarking against these reference labels, we evaluate four frontier LLMs across 11 configurations of reasoning effort and temperature (Table 3). Each configuration is run three times on identical reports. We assess performance using three metrics: mean accuracy against human annotations, self-consistency (the proportion of rubrics yielding identical decisions across all three runs), and majority-voted accuracy at the rubric level.

Temperature and stability. Setting temperature to zero improves self-consistency—e.g., rising from 94.5% to 98.0% for Gemini 3.1 Pro and 91.3% to 94.8% for DeepSeek V4.1 Flash at high reasoning effort. However, stability does not guarantee accuracy: Gemini 3.5 Flash without reasoning achieves 99.0% self-consistency but only 75.62% mean accuracy.

Limited gains from majority voting. Majority voting yields marginal improvements over mean accuracy (≤0.42 percentage points across all configurations), merely shifting accuracy from 84.25% to 84.33% for Gemini 3.1 Pro (low reasoning, zero temperature). We therefore adopt a single judge pass per report for benchmark evaluation.

Based on this comparison, we select Gemini 3.1 Pro with low reasoning effort and zero temperature, which achieves the highest mean accuracy (84.25%) among the tested configurations on these 402 labels. We use this configuration for benchmark grading. This model choice aligns with findings by Peng et al. (2026) in DR settings.

4 Experiments

We evaluate frontier Deep Research agents on LifeSearch to assess their ability to complete real-world tasks and examine agreement between aggregate scores and human report rankings. We first describe the shared execution harness and experimental setup (Section 4.1), then present the main comparative results (Section 4.2). We next examine performance patterns in evidence collection and report synthesis (Section 4.3), followed by an analysis of recurring failures (Section 4.4). Finally, we examine the alignment between aggregate scores and report-level human preferences (Section 4.5).

4.1 Experimental Setup

We evaluate 12 frontier DR agents on all 203 bilingual tasks: GPT-5.6 Sol, Claude Opus 5, Hy4 Preview, GLM-5.3, Qwen3.8-Max, DeepSeek-V4 Pro, Kimi-K3, Doubao-Seed-2.1-Pro, Claude Opus 4.8, Hy3, GLM-5.2, and Gemini-3.6-Flash. We use each model’s default configuration, setting reasoning effort to the maximum supported level where configurable. All systems operate within a shared ReAct-style (Yao et al., 2023) harness that interleaves decisions, tool execution, and feedback from the resulting observations. Agents have access to web search, web fetch, and code execution. Each iteration corresponds to a logical model decision step that may issue multiple tool calls or produce the final answer. A unified system prompt in Appendix B.6 instructs agents to complete the DR task. We use Serper¹¹ 1 https://serper.dev/ as the web-search backend and Jina Reader²² 2 https://jina.ai/reader/ to retrieve web pages.

We evaluate final reports using the score defined in Section 3.4, applying a single grading pass over each report’s rubrics with Gemini 3.1 Pro (Google DeepMind, 2026), as selected in Section 3.5. To account for variability across DR agent trajectories, we conduct three independent runs per task, and report the mean and sample standard deviation of the run-level benchmark scores. We also record iteration counts and model-generated output tokens.

4.2 Main Results

Substantial headroom remains. Table 4 summarizes overall scores, domain performance, and resource usage. GPT-5.6 Sol leads with 63.4%, followed by Claude Opus 5 at 56.1%; all other systems score below 50%. Even the strongest agent leaves 36.6% of the normalized requirement weight unsatisfied. Its three-run mean score also falls below 50% on 41 of 203 tasks (20.2%), indicating that its overall lead does not translate into reliable completion of every research request.

Domain results reveal distinct strengths. GPT-5.6 Sol and Claude Opus 5 rank first and second in all seven domains, but no system exceeds 70% in any domain. Other systems exhibit more varied strengths: GLM-5.3 outperforms Hy4 Preview on everyday facts (51.9% versus 47.1%), while Hy4 Preview leads on academic and professional literature (51.4% versus 49.0%). These differences are descriptive, especially for the nine-task everyday-facts subset.

Output volume does not determine quality. Figure 1(b) relates scores to model-generated output tokens across the full agent run. GPT-5.6 Sol has both the highest score and the largest output volume (51.5k tokens per task). However, Claude Opus 5 and GLM-5.3 generate similar volumes (48.9k and 49.3k), yet their scores differ by 6.9 percentage points. Hy4 Preview reaches a comparable score to GLM-5.3 (49.7% versus 49.2%) with only 33.6k tokens. Thus, greater generation volume alone does not explain performance.

4.3 Performance Patterns

LifeSearch rewards correct, comprehensive answers supported by key evidence and traceable sources. Meeting these requirements involves two complementary capabilities: collecting relevant information through search and synthesizing it into a report that explicitly addresses the user’s needs. The observed performance patterns help explain why resource usage alone is insufficient to characterize research quality.

Evidence collection. Repeated searches and source inspection allow agents to cover complementary aspects of a task, verify facts, and resolve gaps or conflicting evidence. Across the 12 systems, mean interaction count correlates positively with overall score (Spearman’s ρ=0.76). GPT-5.6 Sol averages 86.2 iterations and scores 63.4%, whereas GLM-5.2 averages 11.0 iterations and scores 37.1%. This pattern is consistent with the value of sustained evidence collection. However, iterations include other tool actions, and comparisons across models do not isolate the causal effect of additional searches.

Evidence synthesis. Retrieved information contributes to the score only when the final report presents the facts and supporting evidence needed to satisfy the rubrics. Final-report length, measured in characters rather than tokens generated throughout execution (Appendix C), correlates positively with score (Spearman’s ρ=0.60). Yet Doubao-Seed-2.1-Pro and Gemini-3.6-Flash produce 12,089 and 10,256 characters per report while scoring 43.1% and 34.7%, respectively; Hy4 Preview reaches 49.7% with 9,959 characters. Thus, longer reports can accommodate broader coverage, but their value depends on the relevance, correctness, and completeness of the information they contain.

4.4 Failure Analysis

Our analysis identifies three recurring failure modes: premature termination, temporal scope violations, and incomplete factual coverage. They affect whether an agent produces a final report, whether its evidence is valid for the requested period, and whether the report satisfies the task’s requirements.

Premature clarification. Doubao-Seed-2.1-Pro and GPT-5.6 Sol sometimes ask follow-up questions instead of proceeding with the research task. Because the autonomous harness provides no user replies, these requests can terminate the loop before a final report is produced. The agent consequently leaves the task unresolved and provides no research deliverable for evaluation.

Temporal scope violations. Agents sometimes default to current web data even when the query specifies an earlier cutoff, a behavior particularly noticeable in Hy4 Preview. Retrieved information can be accurate at execution time yet invalid for the requested period. Using updated figures or later events without checking their dates can therefore make an otherwise relevant report inconsistent with the task’s temporal scope.

Missing required details. With around 20 rubrics per task, a fluent summary can address the broad topic while omitting requested entities, numerical values, dates, sources, or edge cases. These omissions leave individual requirements unmet even when the overall narrative appears coherent.

4.5 Human Agreement

We conduct a human ranking study on 30 tasks (15 per language) to examine agreement at the report level. Annotators rank anonymized reports presented in randomized order according to factual accuracy, completeness, and formatting quality. This assessment examines whether aggregate rubric scores capture the relative quality of complete research deliverables.

Table 5 presents a seven-system study, comprising 210 reports. Within this subset, rubric-based and human rankings are strongly correlated (Spearman’s ρ=0.89, Kendall’s τ=0.81), with identical ordering of the top four systems. These results support the alignment of our aggregate rubric-based scores with human assessments of overall report quality.

5 Conclusion

We present LifeSearch, a benchmark for evaluating deep research agents on real-world life and productivity tasks. Spanning seven domains, it comprises 203 bilingual tasks (Chinese and English) with 4,014 expert-annotated rubrics assessing requirement coverage and factuality. Across 12 frontier agents evaluated in a unified harness, the top system scores only 63.4%, with increased output volume failing to consistently ensure higher quality. Our grading protocol is validated against human rubric labels and holistic report rankings. We hope LifeSearch supports the development of more reliable and effective DR agents for everyday life and professional work.

AI use statement

We use ChatGPT solely for language refinement of the manuscript’s text. All conceptual content, experimental design, analysis, and conclusions are developed entirely by the authors. We carefully review the AI-assisted edits to ensure that the meaning and technical accuracy of the original text are fully preserved.

Ethics statement

We hired crowd workers to write the initial query seeds for LifeSearch. All candidate queries undergo privacy screening and anonymization before task construction. Expert annotators responsible for task refinement and rubric annotation are full-time employees and performed annotation as part of their compensated work.

Reproducibility statement

Section 3 details the data collection, expert-guided task refinement, rubric construction, and evaluation procedures. We report judge-model selection experiments and the corresponding hyperparameter settings, including reasoning effort and temperature, in Section 3.5 and Table 3. Appendix B provides prompt templates for intent and quality filtering, domain classification, task rewriting, and rubric-based evaluation in Chinese and English, together with the deep research agent system prompt. Figure 3 summarizes the task distribution and rubric statistics, while Table 6 details the rubric dimensions, annotation fields, and writing requirements. Appendix C documents the evaluation window within a few days in August 2026 and the mean final-answer length for each model.

References

• Chen et al. (2025) S. Chen, P. Moreira, Y. Xiao, S. Schmidgall, J. Warner, H. Aerts, T. Hartvigsen, J. Gallifant, and D. S. Bitterman Medbrowsecomp: benchmarking medical deep research and computer use. arXiv preprint arXiv:2505.14963. Cited by: §1, §2.
• Du et al. (2026) M. Du, B. Xu, C. Zhu, L. Zhang, X. Wang, and Z. Mao Deepresearch bench: a comprehensive benchmark for deep research agents. In International Conference on Learning Representations, Vol. 2026, pp. 42414–42448. Cited by: Table 1, §1, §2.
• Gemini (2025) Gemini Gemini deep research. Note: https://gemini.google/overview/deep-research/ Cited by: §1.
• Google DeepMind (2026) Google DeepMind Gemini 3.1 Pro. Note: Model cardPublished February 19, 2026. Accessed September 20, 2026 External Links: Link Cited by: §4.1.
• Gou et al. (2025) B. Gou, Z. Huang, Y. Ning, Y. Gu, M. Lin, W. Qi, A. Kopanev, B. Yu, B. J. Gutiérrez, Y. Shu, C. H. Song, J. Wu, S. Chen, H. N. Moussa, T. Zhang, J. Xie, Y. Li, T. Xue, Z. Liao, K. Zhang, B. Zheng, Z. Cai, V. Rozgic, M. Ziyadi, H. Sun, and Y. Su Mind2Web 2: evaluating agentic search with agent-as-a-judge. arXiv preprint arXiv:2506.21506. External Links: Link Cited by: Table 1, §1, §2.
• Jin et al. (2025) B. Jin, H. Zeng, Z. Yue, J. Yoon, S. Arik, D. Wang, H. Zamani, and J. Han Search-r1: training llms to reason and leverage search engines with reinforcement learning. arXiv preprint arXiv:2503.09516. Cited by: §2.
• Li et al. (2025) M. Li, Y. Zeng, Z. Cheng, C. Ma, and K. Jia Reportbench: evaluating deep research agents via academic survey tasks. arXiv preprint arXiv:2508.15804. Cited by: §2.
• Mialon et al. (2023) G. Mialon, C. Fourrier, C. Swift, T. Wolf, Y. LeCun, and T. Scialom GAIA: a benchmark for general ai assistants. arXiv preprint arXiv:2311.12983. Cited by: §2.
• Moonshot AI (2025) Moonshot AI Kimi-Researcher: end-to-end RL training for emerging agentic capabilities. Note: Technical report External Links: Link Cited by: §2.
• OpenAI (2025) OpenAI Introducing deep research. Note: https://openai.com/index/introducing-deep-research/ Cited by: §1.
• Peng et al. (2026) Y. Peng, Y. Qi, H. Peng, H. Xia, G. He, X. Shi, R. Xuan, S. Lu, Y. Liu, Z. Hu, et al. Can LLM-as-a-Judge reliably verify rubrics in agentic scenarios?. arXiv preprint arXiv:2606.29920. Cited by: §3.5.
• Sharma et al. (2026) M. Sharma, C. B. C. Zhang, C. Bandi, C. Wang, A. Aich, H. Nghiem, T. Rabbani, Y. Htet, B. Jang, S. Basu, et al. Researchrubrics: a benchmark of prompts and rubrics for evaluating deep research agents. In International Conference on Learning Representations, Vol. 2026, pp. 90447–90472. Cited by: Table 1, §1, §2.
• Singh et al. (2025) R. Singh, S. Joel, A. Mehrotra, N. Wadhwa, R. B. Bairi, A. Kanade, and N. Natarajan Code researcher: deep research agent for large systems code and commit history. arXiv preprint arXiv:2506.11060. Cited by: §1.
• Team et al. (2025) T. D. Team, B. Li, B. Zhang, D. Zhang, F. Huang, G. Li, G. Chen, H. Yin, J. Wu, J. Zhou, et al. Tongyi deepresearch technical report. arXiv preprint arXiv:2510.24701. Cited by: §2.
• Wang et al. (2026a) J. Wang, Y. Ming, R. Dulepet, Q. Chen, A. Xu, Z. Ke, F. Sala, A. Albarghouthi, C. Xiong, and S. Joty Liveresearchbench: a live benchmark for user-centric deep research in the wild. In International Conference on Learning Representations, Vol. 2026, pp. 95096–95143. Cited by: Table 1, §2.
• Wang et al. (2026b) Y. Wang, L. Wang, Y. Deng, K. Wu, Y. Xiao, H. Yao, L. Kang, H. Ye, Y. Jing, and L. Bing DeepResearchEval: an automated framework for deep research task construction and agentic evaluation. arXiv preprint arXiv:2601.09688. External Links: Link Cited by: §1, §2.
• Wei et al. (2025) J. Wei et al. BrowseComp: a simple yet challenging benchmark for browsing agents. arXiv preprint arXiv:2504.12516. External Links: Link Cited by: Table 1, §1, §2.
• Wong et al. (2026) R. Wong, J. Wang, J. zhao, L. Chen, Y. Gao, Zhanglong, X. Zhou, Z. Wang, K. Xiang, G. Zhang, W. Huang, Y. WANG, and K. Wang WideSearch: benchmarking agentic broad info-seeking. In International Conference on Learning Representations, C. Vondrick, B. Hariharan, C. Raffel, L. Pinto, D. Yang, and A. Faust (Eds.), Vol. 2026, pp. 10012–10086. External Links: Link Cited by: Table 1, §1, §2.
• Yao et al. (2023) S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. R. Narasimhan, and Y. Cao ReAct: synergizing reasoning and acting in language models. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023, External Links: Link Cited by: §4.1.
• Yoran et al. (2024) O. Yoran, S. J. Amouyal, C. Malaviya, B. Bogin, O. Press, and J. Berant Assistantbench: can web agents solve realistic and time-consuming tasks?. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 8938–8968. Cited by: §2.
• Zhang et al. (2025) W. Zhang, X. Li, Y. Zhang, P. Jia, Y. Wang, H. Guo, Y. Liu, and X. Zhao Deep research: a survey of autonomous research agents. arXiv preprint arXiv:2508.12752. Cited by: §1.
• Zhong et al. (2026) J. Zhong, H. Zhang, C. Southern, J. Yang, T. Wang, K. Jung, S. Zhang, D. Yarats, J. Ho, and J. Ma Draco: a cross-domain benchmark for deep research accuracy, completeness, and objectivity. arXiv preprint arXiv:2602.11685. External Links: Link Cited by: Table 1, §1, §2.
• Zhou et al. (2024) S. Zhou, F. F. Xu, H. Zhu, X. Zhou, R. Lo, A. Sridhar, X. Cheng, T. Ou, Y. Bisk, D. Fried, U. Alon, and G. Neubig WebArena: a realistic web environment for building autonomous agents. In International Conference on Learning Representations, B. Kim, Y. Yue, S. Chaudhuri, K. Fragkiadaki, M. Khan, and Y. Sun (Eds.), Vol. 2024, pp. 15585–15606. External Links: Link Cited by: §2.
• Zhu et al. (2025) F. Zhu, X. Y. Ng, Z. Liu, C. Liu, X. Zeng, C. Wang, T. Tan, X. Yao, P. Shao, M. Xu, et al. Findeepresearch: evaluating deep research agents in rigorous financial analysis. arXiv preprint arXiv:2510.13936. Cited by: §1, §2.

Appendix A Rubric Design Details

Table 6 expands the overview in Section 3.3 with the annotation fields, authoring requirements, and illustrative examples.

Appendix B Data Collection and Evaluation Prompt Templates

We provide English translations of the filtering, domain classification, and rewriting prompts associated with Section 3.1. The filtering template presents three selected judgments from the original combined prompt, renumbered as 1, 2, and 3. Output keys remain unchanged, while quality labels and category names are translated into English. Placeholders indicate the input query. We also provide the original Chinese and English judge prompt templates used for rubric evaluation in Section 3.4, including the priority field. The judge determines rubric satisfaction; the annotator-assigned weights are applied separately during score aggregation. The search-agent system prompt is included in Appendix B.6.

B.1 Intent and Quality Filtering

The following template combines task quality, search relevance, and implicit event-dependence assessment. Its time-dependence flag distinguishes underspecified event premises from general requests for up-to-date information.

注意转载授权 · 阅读原文