大语言模型能写文献综述吗？研究以102篇文献基准测试六款AI工具

   David M. Schultz\affa,c    Alexandra K. Anderson-Frey\affd    Zhonghua Zheng\affa,c    Catherine McGuire\affe Affiliation: \affaDepartment of Earth and Environmental Sciences, The University of Manchester, Manchester, United Kingdom
\affbDepartment of Computer Science, The University of Manchester, Manchester, United Kingdom
\affcCentre for Crisis Studies and Mitigation, The University of Manchester, Manchester, United Kingdom
\affdDepartment of Atmospheric and Climate Science, University of Washington, Seattle, Washington, United States
\affeResearch IT, The University of Manchester, Manchester, United Kingdom

Abstract

Large language models (LLMs) can be used to write literature reviews, but how good are they at that task? Previous evaluations have tested steps of that task rather than the whole. This study follows the process of using LLMs to write a literature review in four steps: finding the relevant articles, interpreting each one, synthesizing across them, and proposing future work. This study uses a benchmarked literature review that was manually constructed, written, and published—a literature review on the Spanish-plume weather pattern created from a 102-article database derived from a systematic search by Schultz et al. (2025). In the first step, five AI tools searched for those articles. The best tool found only 42 of the 102, and together the five found 57. Of 18 articles with substantial mentions of the Spanish plume, the tools found between 4 and 12. Library access mattered: With a university library login, two coding agents found 82 and 79 articles, against 42 and 15 without. In the second step, three tools were then given PDFs of all 102 articles. All three correctly identified the 18 substantial articles, but they misclassified between 40% and 79% of the 84 passing mentions as substantial. In the third step, the same three tools each wrote a synthesis of the literature based on ten questions. Two expert reviewers scored the syntheses without knowing which tool had written which. They found that the syntheses were largely accurate, but found little insight from them. In the fourth step, all three tools proposed topics broader than just the Spanish plume, with two of the tools proposing future research topics close to those in the benchmark review. Our results show that no single tool was the best for all purposes. We make eight recommendations for researchers seeking to use AI tools in the literature review process: search with subscription access and review where and why the tool was blocked; treat an AI-written synthesis as a list of claims to verify, not as a draft; have an expert review the output; check whether some publishers’ journals are missing from the results; check whether older articles are missing; judge the importance of each article yourself; write a detailed prompt; and do not take a tool’s confidence as a sign that it is right. Although these tools can assist a researcher in finding and reading the literature that they can access, the tools tested in this study did not synthesize the literature effectively.

1 Introduction

Creating a literature review is time-consuming, and the parts that take the most time are also the ones that cannot be skipped: finding all relevant sources, reading each article carefully to obtain an accurate reflection of their results, synthesizing across the body of literature, and deciding what future research directions are worthwhile. Large language models (LLMs) now offer to do all four steps. Authors are already using them (Lieberum et al., 2025, e.g.,), journals are already publishing the results (Schiller, 2024; Scherbakov et al., 2025, e.g.,), and researchers have begun to describe recognizable failure modes in AI-written reviews (Zhao et al., 2025; Poe and Dryer, 2026, e.g.,). Thus, we are in what has been called the “LLM-supported” era of reviews (Hauser, 2025). But, how effective are these AI tools at writing literature reviews?

Previous evaluations have tested only parts of this task such as the references a tool generates, the reliability of its answers, and the fidelity of its citations (Tang et al., 2025; Wu et al., 2026; Zhang et al., 2026, e.g.,). At the time of the writing of this article (October 2026), the results are mixed. For example, in a systematic review of LLMs conducting systematic reviews in health research, authors evaluated LLM use as promising in 54% of the 37 studies, neutral in 24%, and nonpromising in 22% (Lieberum et al., 2025). More specifically, Zhang et al. (2026) used GPT-4 Turbo to answer eight specific questions that were manually checked against the actual answers. They found 86–98% accuracy. In contrast, Wu et al. (2026) evaluated three LLMs and found accuracy rates of 20–88%. We are unaware of studies that compare different tools on the same corpus of literature and follow the steps in the process of reviewing the literature and writing a literature review.

Moreover, evaluating whether the tools deserve our trust is harder than it looks. A literature review may cite real unhallucinated articles and still misinterpret them (Tang et al., 2025; Wu et al., 2026, e.g.,). Testing the ability of an AI tool to synthesize requires a corpus whose contents are known in full (so that omissions can be counted) and reviewers who know the literature well enough to spot a plausible-sounding error. Those two things rarely coincide in recent studies. Our present study has both.

Others have found success by breaking the process of writing literature reviews down into steps that can be constructed and evaluated separately (Agarwal et al., 2024, e.g.,) and by comparing AI tool–generated literature reviews to ones written by humans (Mostafapour et al., 2024, e.g.,). We also follow that approach (Fig. 1). To evaluate different LLMs across four steps in the literature-review process, we choose a baseline scientific literature review that has been human-written and already been published. Schultz et al. (2025) conducted a systematic search of the literature on the Spanish plume, a weather pattern associated with severe convective storms in western Europe. Their corpus contained 102 articles (Schultz et al., 2024), classified according to how each used the term and how central it was to the article. Then, two coauthors of the present study (DMS, the lead author of that review, and AKAF, a convective-storms expert, but not a coauthor on that review) evaluated LLM-written syntheses of the same corpus, blinded to which tool had produced which. We then evaluate the success of six AI tools through four steps in the literature-review process using various quantitative and qualitative metrics.

This article is organized as follows. Section 2 describes the benchmark review, the corpus, and the tools. Sections 3–6 take the four steps in turn, each with the previous work that bears on it, the method, the results and what they mean. Specifically, section 3 asks whether an LLM can find the relevant articles. Section 4 asks whether it can interpret each one. Section 5 asks whether it can synthesize across them. Section 6 asks whether it can propose concrete future work. Section 7 draws recommendations for researchers using these tools, Section 8 discusses the limitations, and Section 9 concludes this article.

2 Data and methods

2.1 The benchmark review

Schultz et al. (2025) searched Google Scholar, Web of Science, American Meteorological Society journals, and Royal Meteorological Society journals for the exact phrase “Spanish plume”, and assembled 102 peer-reviewed articles published between 1968 and 2023. They classified each by how the term was cited. Eighty-four articles mentioned the plume in passing. Fourteen, which we call informative, used it as an integral part of their argument. Four, which we call essential, were specifically about it: Carlson and Ludlam (1968), who first described the mechanism; Morris (1986), who popularized the name, and both Gray and Marshall (1998) and Lewis and Gray (2010), who studied the climatology and synoptic weather patterns associated with Spanish plume events. Together, these 18 articles represented the substantial articles. The review also recorded, for each article, whether the term referred to the elevated dry airstream, to the broader synoptic weather pattern, or to both, and it documented the confusion that has followed from that ambiguity.

Three properties make the Schultz et al. (2025) review a good benchmark. First, the corpus is bounded, so the share of articles found is measurable. Second, the articles span several publishers with different access rules, allowing us to examine what these AI tools can access. Third, one of the central findings of the review is a terminological one: The same term “Spanish plume” has been used for different things, which is the kind of ambiguity that requires careful understanding to catch, making it a good test of the AI tools.

2.2 The corpus

The 102 articles were retrieved as PDFs and assigned identifiers SP001–SP102 in order of publication year. Every file was checked for a machine-readable text layer; none required optical character recognition. The corpus and its metadata were prepared before the synthesis experiment (section 5) and kept unchanged afterwards. The benchmark review (Schultz et al., 2025), its supplementary database (Schultz et al., 2024), and its classifications were withheld from every tool in every experiment.

2.3 The tools

Six tools were used, consisting of three types: three online assistants, two coding agents, and a retrieval-grounded assistant. First, an online assistant answers in a chat interface and searches the web through the product’s own tooling: ChatGPT Deep Research, Gemini Deep Research, and Claude’s web interface. All three were run in their deep-research modes (Deep Research in ChatGPT and Gemini, Research in Claude), in which the assistant plans and carries out many web searches before writing a report. Second, a coding agent runs on the user’s own machine with a terminal, a file system, and a browser that carries the user’s sessions: Codex and Claude Code. Third, a retrieval-grounded assistant answers only from documents uploaded to it: NotebookLM, renamed Gemini Notebook in July 2026. NotebookLM was also asked to search, but that search used the benchmark’s public dataset despite an instruction not to, so it was not scored (Section 8). Table 1 lists the underlying model, the plan, and which of the four tasks each tool performed. All six were used on the entry-level paid plan of their provider, and wherever a product offered a reasoning-effort setting, it was set to its maximum for every task. In Sections 4–6, “Claude” refers to Claude Code.

3 Can LLMs find relevant articles?

3.1 Previous work

Jain et al. (2024) found that five different LLMs were able to find 4–48% of the articles from the total set of references from all five LLMs combined. Poe and Dryer (2026) described a “missing middle” in AI-written literature reviews: a handful of foundational articles from the 1980s and early 1990s, a cluster of very recent ones since 2023, and little in between (i.e., mid 1990s to early 2020s), which they attributed to publisher firewalls. That claim is testable only against a literature whose full extent is known, and Section 33.4 tests it. The mechanism they proposed, that a model finds what is openly available and stops at the paywall, also predicts that what a tool finds should depend on the publisher and the access route rather than on the model, which Section 33.3 examines directly.

3.2 Method

Five tools were asked to find the peer-reviewed literature that mentions the exact phrase “Spanish plume”, using the same prompt, without access to the benchmark review or anything derived from it: ChatGPT Deep Research, Codex, Gemini Deep Research, Claude (web interface), and Claude Code. The prompt named the four databases used by the benchmark, required each result to carry evidence that the phrase appears in the article, and prioritized finding as many articles as possible. Results were matched to the corpus by DOI and normalized title. The saved search results were produced in late July and early August 2026. Institutional authentication and browser state were not controlled uniformly across these five runs, so the comparison describes what each tool reported in practice rather than a matched test of model capability. The Codex list used here is the 8 August cookie-cleared run, not its earlier search report. An article was counted as found only when the tool’s report named it; a total that a tool reported without naming the articles was recorded separately.

A paired test then isolated the effect of access. Codex and Claude Code were each run twice with an identical prompt: once with browser cookies cleared and no institutional session, and once after the first author signed in to the University of Manchester’s publisher and database subscriptions.

3.3 Results

All five tools found fewer than half of the 102 articles (Table 2). Codex found 42, ChatGPT Deep Research 40, Gemini Deep Research 16, Claude Code 15 and Claude’s web interface 7. Gemini reported that 35 articles passed its screening of 277 records, but its report named only 16 of them, so the other 19 could not be checked against the benchmark. Together, the five tools found 57 different articles. The other 45 were not found by any tool, although some of them may be among Gemini’s 19 unnamed articles.

Figure 2 compares the articles found with the benchmark’s year distribution. The tools found 78% of the articles published from 2020 onward but only 48% of those from 1995 to 2019, so their results were weighted toward recent work.

The share of articles found varied by journal (Table 3). Articles in American Meteorological Society journals are free to read one year after publication, and the tools found 10 of the 11 in the corpus, including all 5 in Weather and Forecasting and 3 of the 4 in Monthly Weather Review. The tools found 13 (37%) of the articles in Weather, which holds 35 of the 102, and 2 of the 3 in Meteorological Applications. Only 2 of the 11 pre-2005 Weather articles were found, against 7 of the 11 pre-2005 articles in other journals. This bears on the “missing middle” that Poe and Dryer (2026) described, which Section 33.4 discusses.

The tools met different barriers along the way. Web of Science was unavailable to both coding agents without a university subscription. Google Scholar interrupted Codex with a CAPTCHA and Claude Code after 15 results; AMS blocked Claude Code entirely but admitted Codex. Access was not the only problem; search engines also differ in what they expose to search: Web of Science covers only titles, abstracts and keywords, whereas Google Scholar covers the full text. For “Spanish plume”, Web of Science returned 11 results and Google Scholar 183.

In the paired test, Codex went from 42 benchmark articles before login to 82 after, and Claude Code from 15 to 79 (Fig. 3). Codex found none of the pre-1995 articles before login and 75% after.

Without library access, Codex found 28% of the articles published before 2000 and 59% of those from 2020 onward, and Claude Code 6% and 33%. After login, Codex found 72% and 74%, and Claude Code 39% and 93% (Fig. 3). Figure 4 shows the same comparison by publisher. Most of the gain came from Wiley’s journals, where Codex went from 21 of the 62 articles to 53 and Claude Code from 12 to 53. Before login, Claude Code found no articles from five of the seven publisher groups.

3.4 Discussion: Access, not capability, limited what the tools found

The search experiment reveals constraints imposed by publishing infrastructure as well as differences among tools. Whether a tool found an article depended on two things: whether it could reach the article, and whether it could search its text. Tools of the same type found very different numbers of articles: the online assistants found 40, 16 and 7, and the coding agents 42 and 15. Because the tools differ in more than their underlying model, including how they search and which websites admit them, these differences cannot be attributed to the models alone. Articles that mention “Spanish plume” only in their bodies cannot be reached by a search that indexes metadata alone.

The tools were held back by four barriers outside their control. First, Web of Science requires a subscription. Second, even with a subscription, Web of Science searches only an article’s title, abstract and keywords, not its body text. Third, part of the literature exists online only as images. Fourth, Google Scholar and the American Meteorological Society limit automated access. The same prompt therefore met different barriers depending on which tool carried it. Articles in American Meteorological Society journals are free to read one year after publication, yet Claude Code found none of the 11 before login, because the society’s website refused its automated requests. Codex, which the site admitted, found 8 (Fig. 4). Wiley, by contrast, serves Weather articles from before about 2005 as metadata and a reference list, with the article body available only as a scanned image, so the phrase cannot be matched inside them.

Login to the university library nearly doubled the number of articles Codex found (42 to 82) and substantially raised the number Claude Code found (15 to 79). After login, Codex found a similar share of articles from every period, but Claude Code still found only 39% of those published before 2000 (Fig. 3). In other words, most of what the tools missed before login was a matter of access: Given the same subscriptions a researcher has, the tools were able to find far more.

Poe and Dryer (2026) argued that AI-written reviews cited a few decades-old foundational articles (often from the 1980s and early 1990s) and a run of very recent ones (from 2023 to 2026), skipping the literature between (from the mid-1990s to the early 2020s), because the tools could not get behind publishers’ firewalls. In the present study, the tools found 21 of the 27 articles from 2020 onward (78%), compared with 34 of the 71 from 1995–2019 (48%). They found three of the four essential articles; the exception, Gray and Marshall (1998), was in Weather. These values are minima, because Gemini did not name all the articles it reported. The corpus contains only four articles before 1995, limiting the comparison to the results of Poe and Dryer (2026).

Bot controls, subscriptions, indexing depth, and digitization all shape what an AI agent can find. The practical implication is that a coding agent running inside an authenticated institutional session can find different articles than the same agent on the open web. Future evaluations of LLM literature searches should control that environment and require a complete, auditable article list from every tool.

4 Can LLMs interpret articles?

4.1 Previous work

Wu et al. (2025) asked whether the sources a model cites actually support the claim they are attached to, and automated that check. They found that 50–90% of the responses from seven LLMs were not fully supported by the sources cited, and that even for GPT-4o with web search, about 30% of individual statements were unsupported. van Dort and Heuss (2026) traced the mechanism inside a retrieval-augmented model and found that the decision to attach a citation is separable from whether the cited passage is the source of the statement, so a well-formed citation is not by itself evidence of grounding. Both of these studies bear directly on the present study, in which every quotation a tool reported was checked against its source. Zhang et al. (2026) used GPT-4 Turbo to review the microphysics literature for a numerical weather prediction model, with structured extraction as an intermediate step, which is the form this task takes here. They found that GPT-4 Turbo could answer eight questions about journal articles with 86–98% accuracy against human-checked answers.

4.2 Method

Schultz et al. (2025) built their review on a database of the literature, one row per article, assembled before any of the review was written. Three tools were asked to build the same kind of database. Given the corpus and a locked 17-column schema, each produced one row per article, with no synthesis: the role of the term in the article, the sense in which it is used, the airstream properties and diagnostic variables reported, the origin and transport claims and their evidence, the plume configuration [i.e., classic or modified Spanish plume, as per Lewis and Gray (2010)], capping and initiation mechanisms, hazards, whether a prefrontal trough is described, a supporting quotation with its page, and stated limitations. Codex, NotebookLM and Claude each built a database from the same locked prompt file, in a context holding only the corpus, with no web access and no sight of the benchmark. The rows were scored against the fields of the benchmark database that record the same information, with the benchmark database treated as the gold standard: Any departure counted as an error. Every quotation was then checked against the supplied PDF by an independent script, which searched the page the tool named and then the whole document under three text-extraction modes, ignoring hyphenation, line numbering, and spacing so that only a genuine departure from the source counted as a failure. Details of the quotation check are given in the public repository (see the data availability statement).

4.3 Results

All three tools returned a database covering all 102 articles in the required order with no blank cells. All of Codex’s and Claude’s quotations appeared word for word in the articles they cited. Codex gave a quotation for 100 of the 102 articles, and every one appeared word for word on the page it named. For the other two it gave none, marking them as having no mention in the main text; in one of them, the phrase appeared only in the reference list. Claude gave a quotation for 101 articles, all of them real text from the article cited, with the page number correct for 98 and wrong for 3.

NotebookLM’s quotations were the least correct. Of its 101 quotations, 94 appeared word for word in the article it cited; the other 7 were taken from the article but altered its wording, dropping or changing words without marking the change. Its page numbers were less consistent: although the prompt asked for the page of the PDF, it mixed PDF pages with printed journal pages, and many were on neither. Of its 94 word-for-word quotations, 42 were on the page it gave, counting either the PDF or the printed page, and 52 were not.

Scored against the benchmark database, all three tools classified the 18 substantial articles correctly and misclassified many of the 84 passing mentions (Table 4 and Fig. 5). The errors were on the passing mentions: Codex misclassified 79% of them as substantial, NotebookLM 77% and Claude 40%. Cohen’s κ, which measures agreement with the benchmark beyond what chance would produce, was 0.09 for both Codex and NotebookLM (i.e., little better than chance) and 0.34 for Claude (i.e., fair agreement). All three tools had a recall of 1.00 for the substantial articles but a precision of only 0.21 to 0.35, because they classed so many passing mentions as substantial. Recognizing an article about the Spanish plume was therefore easy for all three tools. Recognizing one that mentions it only in passing was harder: Claude was appreciably better at it than the other two, but it still misclassified 40% of the passing mentions.

The tools also recorded whether an article used “Spanish plume” to mean the airstream, the synoptic weather pattern, or both, and they agreed with the benchmark database for about two-thirds to three-quarters of the articles (Table 4). The disagreements went both ways: Codex and Claude assigned the airstream meaning to many more articles than the benchmark database did, and NotebookLM to many fewer. Where the benchmark database recorded a classic or a modified Spanish plume, the tools mostly agreed with it.

4.4 Discussion: Accurate extraction, unreliable interpretation

The tools extracted text from the articles accurately, but they were unreliable at judging what an article did with the term. All three tools recognized the 18 articles that discuss the Spanish plume substantially, yet they also classified between 40% and 79% of the passing mentions as substantial, implying the tools often mistook a mention of the term for a discussion of it. For a literature review, making this distinction matters, because the tools would overstate the extent to which the literature discusses the topic in depth.

Quotation accuracy depended on the tool. Codex and Claude quoted word for word in every row where they gave a quotation, whereas 7 of NotebookLM’s 101 quotations altered the article’s wording. NotebookLM was also the tool whose two quotations in its synthesis were misattributed (Section 55.3). The database made these checks easy. Each row gave a quotation with its article and page, so a script could check all of them. In a written synthesis, the quotations were spread throughout the text, and each one had to be found and checked by hand. A researcher who asks a tool for a synthesis may therefore want a database from it, as well, because the database can be checked automatically.

5 Can LLMs synthesize across articles?

5.1 Previous work

Tang et al. (2025) evaluated LLMs on reference generation, abstract writing and review composition. They found that at most 52% of the references the LLMs generated matched a real article. Wu et al. (2026) proposed a benchmark for the reliability of LLMs in searching for and reading papers. They found that three LLMs missed more than 60% of the relevant literature when asked to find articles on a topic. Abdulhai et al. (2026) found that when AI tools were asked only to correct grammar, they shifted the meaning of the text they edited, which is the failure a synthesis risks at corpus scale. These three studies measured whether a model’s output was true and whether its citations were real. None of them asked domain experts to judge a cross-article synthesis against a complete manual review of the same literature.

5.2 Method

NotebookLM, Codex, and Claude Code received all 102 PDFs and the same locked prompt. They were asked to write a critical synthesis, not an article-by-article summary, organized around ten themes:

1. 1.

   evolution of the term and concept;

2. 2.

   airstream versus synoptic-pattern usage;

3. 3.

   thermodynamic and moisture properties;

4. 4.

   claimed geographical origin and transport pathways;

5. 5.

   evidence used to support origin claims;

6. 6.

   capping, instability, and convection-initiation mechanisms;

7. 7.

   hazards, locations, and event types;

8. 8.

   areas of agreement;

9. 9.

   disagreements, unsupported assumptions, and uncertainty;

10. 10.

    remaining research gaps.

The prompt required a source and a page for every claim, forbade outside knowledge and web access, and instructed the tool to write “Not established by the provided sources” where evidence was absent. The prompt also asked each tool to label statements as a direct finding, an author interpretation, or a cross-article inference. Four of the ten themes encoded concerns raised by the benchmark review itself, so this was a guided synthesis test rather than an open-ended one. The tools were not asked to discover the problems in the literature, only to discuss them.

The prompt set the same rules for all three tools:

1. 1.

   Cite the supporting source and its one-based PDF page as [SPxxx, PDF p. N].

2. 2.

   Label each statement as a direct finding, an author interpretation, or a cross-article synthesis inference.

3. 3.

   Weight analytical studies more heavily than passing mentions, and do not treat an appearance in a reference list as evidence that an article discusses the plume.

4. 4.

   Do not convert repeated unsupported claims into established facts, and do not describe anything as consensus unless the supplied sources demonstrate it.

5. 5.

   Never guess a page number, and where the evidence is insufficient, write “Not established by the provided sources.”

6. 6.

   Close with a claim ledger giving, for every theme, the claim, its supporting sources, any contradicting or qualifying sources, the strength of the evidence, and the residual uncertainty.

The three syntheses were anonymized as Syntheses 1, 2 and 3, with the key held by the first author and withheld from both reviewers (1 was NotebookLM, 2 was Codex, and 3 was Claude). Citation formatting was standardized across the three so that style could not identify a tool; claims, hedges, identifiers and page numbers were left untouched. Two coauthors (DMS and AKAF) scored each synthesis independently, theme by theme, on five criteria developed jointly: writing quality and readability, depth and breadth, veracity, insight, and usefulness to a researcher conducting a review. Each criterion was rated from 1 (strongly disagree) to 5 (strongly agree), giving 50 ratings per synthesis per reviewer and 300 in total. Reviewer A is the lead author of the benchmark review (DMS); Reviewer B is an independent researcher in severe convection who was not involved in that study (AKAF). Both also returned written comments. Reviewer A additionally checked every direct quotation in Syntheses 1 and 2, and those in the first two themes of Synthesis 3, against the source PDFs.

5.3 Results

The three syntheses differed in size. NotebookLM wrote 3624 words across the ten themes and cited 41 distinct corpus articles; Codex wrote 3919 words and cited 27; Claude wrote 7480 words and cited 79. None of these numbers predicted the expert ratings.

Table 5 gives the mean rating per criterion. Veracity was rated higher than insight in all six combinations of reviewer and tool, but it was not always the highest-ranked criterion, and insight was not always the lowest. Reviewer B scored veracity at 3.6 to 3.8 and insight at 2.3 to 2.5 for every tool, and wrote that the outputs were “fine, but quite uninsightful” and “just a summary without any insights.” Reviewer A described the same tendency: “Like undergraduates sometimes do, the LLM offers text that is not answering the original question.” The tools reported the literature; they did not synthesize it.

The reviewers also converged on a failure mode. Theme 8 asked for areas of agreement in the literature, and all three tools answered with more confidence than the corpus supports. Synthesis 1 asserted “absolute consensus” and called the plume “the primary driver of Europe’s most severe summertime convective hazards”; Reviewer B challenged both phrases, and Reviewer A noted that all three tools “uncharacteristically use strong language and express confidence” on that theme, and asked whether a model recognizes degrees of certainty at all. The prompt had explicitly forbidden describing anything as consensus unless the sources demonstrated it.

Reviewer A’s quotation check produced the sharpest single result. Codex used no direct quotations. NotebookLM used two, and both were wrong. One quotation was attributed to Fery and Faranda (2024), a sentence that does not appear in the article but appears word for word in a referee’s comment on their manuscript, at a page number the article does not have. The other quotation was from Schultz (2005), with the wrong page number. Although it was outside the quotation marks, NotebookLM changed the word “mechanism” in the original to “mechanics” in its synthesis, making the sentence nonsensical.

Claude quoted extensively, and every quotation checked was accurate, although two multi-author articles were cited by their first two authors rather than as “et al.” (e.g., “De Meutter and Gerard” rather than “De Meutter et al.”). The databases gave the same picture: NotebookLM was the only tool that misquoted, and it did so in both tasks (Section 44.3). The reviewers also found errors that no citation audit would catch: One synthesis abbreviated “capping inversion” as CIN, which denotes “convective inhibition” instead; one described a warm conveyor belt bringing cooler air instead of warmer air; and one described the plume as a warm, moist layer. Several articles in the corpus describe it that way, but the description is inconsistent with the original definition of Carlson and Ludlam (1968), in which the plume is warm, dry air above a moist layer. The synthesis therefore reproduced an error that is present in the literature. Two tools invented or altered terminology for “underrunning” (i.e., a process by which warm moist air flows under the capping inversion and yields the potential for convective storms to form), renaming it “undercutting” and coining the phrase “edge-of-lid escape”. Reviewer A observed that none of the three recognized that Spain lies on the trajectory path from Africa, a point central to the debate about the origin of the Spanish plume airstream.

NotebookLM produced ASCII art to help answer some of these questions of varying quality and utility. In one case, the figure could have been more effective as a table. In the second case, NotebookLM tried to produce an atmospheric sounding profile of the atmospheric layers commonly associated with the Spanish plume. This image did not translate well through ASCII art. The third case was unintelligible. It was not possible to understand what was being graphed nor what it was trying to convey. No explanation of the figure or figure caption was included.

NotebookLM and Claude generally produced readable text. In contrast, the writing with Codex was the most stilted and awkward of all three LLMs (e.g., unusual sentence structure, unusual word choice), often to the point of being a negative and weakening its authority even when it did say something correct or clever. Codex created enumerated or bulleted lists more commonly than the other two. These lists were not necessarily things that needed ordered by a list, however.

5.4 What the syntheses cited

All three tools cited all four essential articles. Of the fourteen informative articles, Claude cited 14, NotebookLM 12, and Codex 9 (Table 6). As a share of each tool’s own citations, the eighteen substantial articles were 48% of Codex’s, 39% of NotebookLM’s and 23% of Claude’s. Citing more articles did not mean citing the right ones, however. Even though Claude cited all 18 substantial articles, it did so by citing 77% (79 out of 102) of the total corpus. In contrast, Codex cited the fewest, 26% (27 out of 102), but had the largest share of its citations on the 13 substantial articles it did cite.

Codex’s five omissions among the informative articles were published in 2005, 2020, 2022, 2023 and 2023. It cited seven of the eight informative articles published before 2020 and two of the six published from 2020 onward (Fig. 6). Figure 7 shows the three citation distributions by year of publication. Codex’s lean toward older articles held across all its citations: the median year of the articles it cited was 2012, against 2016 for NotebookLM, 2017 for Claude and 2015 for the corpus. NotebookLM cited the oldest and newest articles more often than those in between, and Claude cited a larger share of the articles the more recent they were.

5.5 Discussion: Accurate but not insightful, partly by design

Both reviewers identified the same concerns about the tools. The syntheses got things right and did little with them. Some of that was due to the tools themselves. Some of it was due to the prompt, for which small changes have been known to produce dramatically different results (Felizardo et al., 2025). Every grounding rule in the prompt penalized an unsupported statement; none rewarded a justified interpretation. The prompt regulated quotation heavily (exact wording, verified pages, no repair), but never asked for quotation directly. Thus, quoting became the safest way to comply, whereas paraphrasing became a veracity risk. Only one of the three labels, “cross-article synthesis inference”, allowed a tool to reason across articles, and the prompt did not define it. Each coding agent wrote its own definition into a rules file; NotebookLM had none. Both reviewers independently found the labels unclear. Because the prompt was designed mainly to prevent errors, the tools played safe: Their syntheses were accurate and offered little insight, as the reviewers’ scores show.

Reviewer B wrote the following about one of Codex’s answers: “A lot of plausible-sounding discussion that kind of circles around a whole lot of nothing.” A further point was raised by Reviewer A who wrote, “When an LLM hits upon clever insights, but then says something stupid in the next sentence or paragraph, it makes you wonder about the randomness of it all and reminds you that they are ultimately just glorified autocomplete tools. There is no guarantee that they are on a “hot streak” or have “developed insight into the problem”. …sometimes you can’t tell whether the insight is clever or not. As per Spinal Tap: ‘it’s a thin line between clever and stupid.’”

5.6 Discussion: Breadth is not quality

Claude cited 79 articles, NotebookLM 41, and Codex 27. Reviewer A ranked Claude first, NotebookLM second, and Codex third, whereas Reviewer B ranked Codex first. This was not the order of their length: Codex wrote more words than NotebookLM. Codex covered 13 of the 18 substantial articles with 27 citations; Claude covered all 18 with 79. If completeness against the essential literature were the goal, Claude achieved it, but largely by citing most of the corpus, and Reviewer A noted that its length meant “more opportunities to get it wrong.” A review that cites everything has not made the judgment a review is for.

Wu et al. (2026) reported fabricated responses from an older version of Gemini. Our NotebookLM results also revealed problems with the accuracy of the quotation and the attribution of the source. Similarly, Abdulhai et al. (2026) found that LLMs changed the meaning of a text even when asked only to correct its grammar, and NotebookLM’s seven altered quotations are an example. We found no invented references in the five scored searches or in the syntheses: Every article that a tool named in them exists. The excluded NotebookLM search did report one article that does not exist, with a DOI that belongs to an unrelated paper. We did find incorrect citation details, however, including wrong titles attached to real articles in two of the search reports, wrong page numbers, and a quotation attributed to an article when it came from a referee’s comment.

6 Can LLMs propose future work?

6.1 Previous work

Of the four tasks, this is the least studied and the hardest to score. A proposed research gap is not right or wrong in the way a citation is, so it can only be judged by researchers who are experts in the field. Zhao et al. (2025) conducted an evaluation that scored the research directions proposed by GPT-4 against the judgment of humans. They found that applying a structured series of prompts produced a literature review comparable to that written by humans and one that could identify research gaps.

6.2 Method

The tenth theme of the synthesis prompt (Section 55.2) asked each tool to identify the remaining research gaps in the literature, which is the fourth of the project’s questions. The answers provided by the AI tools were anonymized with the rest of the synthesis and scored blind by both reviewers on the same five criteria, giving ten ratings per tool for this theme.

6.3 Results

NotebookLM listed four gaps and presented each as one that the articles themselves had named: that convection starting overnight above the surface is poorly understood, that prefrontal troughs have not been verified with observations, that small-scale cloud processes need better modelling, and that the effect of climate change on Spanish plume events is uncertain.

Codex presented its gaps as its own inferences from comparing articles. It proposed defining the Spanish plume, establishing the origin of the air with trajectories across many events, studying how the airstream is heated and modified over the Iberian plateau, testing how convection is initiated, studying plume events that produced no storms, building comparable records of the hazards, studying the link to meteotsunamis, and attributing changes to climate change.

Claude characterized its gaps by where they came from. It listed gaps that authors in the corpus had stated, including the origin of convergence lines and the overnight initiation of convection. It then listed gaps it inferred from what the corpus did not contain: a test for deciding whether an event is a Spanish plume, the origin of the air across many events, the inconsistency between articles that call the dry air aloft the plume and those that call the moist air beneath it the plume, a comparison of the proposed initiation mechanisms, trends in plume events, and the verification of forecasts of the Spanish plume.

Reviewer A commented that all three tools “pull in gaps in research topics that are larger than the Spanish plume alone”, such as microphysical modelling and coastal coupling, that Claude’s answer stayed the most focused, and that it was the only one to say where its gaps came from. He rated Claude’s answer highest and NotebookLM’s lowest. Reviewer B rated the three answers about equally, noting that NotebookLM was “fine, but quite uninsightful.”

6.4 Discussion: Relevant gaps, but not confined to the topic

The three answers differ in the way that they present and interpret the research needs. NotebookLM only repeated gaps that authors had already stated, whereas Codex and Claude also worked out gaps from what the literature lacked, which is what proposing future work requires.

The gaps that Codex and Claude proposed are close to the benchmark review’s own recommendations. The review recommends keeping to the original definition of the Spanish plume by Carlson and Ludlam (1968) because later articles applied the name incorrectly to a humid near-surface airstream, recommends calculating trajectories explicitly, suggests studying plume events that do not produce storms, and identifies the role of the Iberian terrain and the initiation of convection as its two research opportunities (Schultz et al., 2025). Codex and Claude reached versions of each of these. NotebookLM reached one (i.e., the initiation of convection) and only for the overnight convection.

The weakness was focus. All three tools proposed gaps on subjects broader than the Spanish plume, as Reviewer A noted. Proposing future work also depends on the step before it: A tool can only identify what is missing from a literature if it has understood what the literature contains. These results rest on one answer from each tool, scored by two reviewers.

7 Recommendations

This study compared the results of various AI tools against a manually constructed benchmark literature review. That comparison showed weaknesses that a researcher using these tools could not see from the tools’ output alone. Someone using LLMs to construct a literature review should therefore be aware of the following issues and consider the following recommendations.

1. 1.

   Search with subscription access to improve the results. Review the search results to see where and why the tool was blocked, by auditing its search record.

2. 2.

   The tools reached some publishers’ articles far more readily than others’. A gap in a tool’s results may therefore reflect lack of access rather than the lack of literature. Thus, searches should also be conducted on publishers’ websites and literature databases directly.

3. 3.

   Recent articles were found by the tools more often than older ones. Older work may therefore be missing even when it exists, so search for it separately.

4. 4.

   Judge the importance of each article yourself. Although these tools identified most, if not all, of the most important articles on the Spanish plume, there is no guarantee that such thoroughness would be expected on another topic. Moreover, the high number of articles reported to be important by the tools, but had mere passing citations to the term “Spanish plume”, indicates that a tool cannot reliably characterize an article’s importance.

5. 5.

   The prompt shapes the answer, so more detailed information about the inclusion/exclusion rules for finding or more detailed information about the character of the answers expected during interpretation generally makes for better results.

6. 6.

   Check the claims and quotations generated by a tool’s synthesis against the original sources. As a common disclaimer on generative AI tools reads, “AI-generated content may be incorrect”.

7. 7.

   A tool will not point out its own weak spots. It tends to sound confident even when it is wrong.

8. 8.

   Expert review by humans is still needed. Some of a tool’s errors can only be caught by someone who knows the subject.

None of this means the tools are not useful. They can find many of the relevant articles and summarize them reasonably accurately, which could save time. But their searches have gaps, their judgments are unreliable, and their confidence is not a guide to accuracy, so their work has to be checked before it is used.

8 Limitations

This study tested one topic and one set of 102 articles, and each tool was run once on each task. Differences between tools could therefore reflect chance variation between runs, which was not measured. New versions of these AI tools are being released regularly and in short order. Our results describe the output from versions of these six AI tools available in July and August 2026. Those researchers looking to see if the results change over time with future versions of these same tools can find the prompts and data in the public repository (Al Kalbani et al., 2026). Although the tools were instructed not to use the benchmark review, and the results suggested they did not, we cannot know whether the underlying LLMs had been trained on it, as the review was published more than a year before our tests were performed.

The search results have several limitations. The five tools were not run under identical conditions, because login and browser state were not controlled in the same way for each. In the login test, logging in gave the two coding agents subscription access and also let them past websites that had blocked them, so the test cannot separate those two effects. A sixth search, with NotebookLM, was excluded because its sources included the benchmark’s public dataset despite an instruction to exclude it. The benchmark has only four articles from before 1995, which limits what can be said about older literature.

The databases built by the tools were scored on 4 of their 17 fields, the ones that the benchmark database also records. Quotations and their page numbers were checked against the original articles, but the remaining fields, such as the origin of the air and the hazards could not be checked. The benchmark database was treated as correct, so any error in it was counted as an error by the tool. In two fields, Claude added a short justification after its answer, which the other tools did not; only the answer itself was scored.

The synthesis prompt told the tools what to write about, rather than leaving them to write openly, so we did not test whether they would have found the main problems in the literature on their own. Proposing future work was also only one of the prompt’s themes, so each tool answered it once.

The results also depend on the prompts. Each task used a single prompt, and no other wording was tested. The synthesis prompt was written to prevent errors and did not ask for interpretation, it listed three labels without defining them, and it set no limit on length. The low ratings for insight, and the differences in length between the syntheses, may therefore reflect the prompt as much as the tools. Different prompts could yield different results.

The two reviewers differed. Across the 150 ratings each gave, they agreed exactly on 24% and within one point on 67%, Reviewer A scored 0.94 points higher on average than Reviewer B, and they ranked the three tools differently. Two reviewers are too few to assess statistical significance, and the results would be subjective anyway. Reviewer A led the writing of the benchmark review and was potentially biased by being too close to the subject matter. The ratings are therefore given separately for each reviewer, and the conclusions rest on what both reviewers found, not on any ranking of the tools. This result is perhaps not surprising and is comparable to results from other studies that reviewers often do not agree with each other (Ernst et al., 1993; Schultz, 2010, e.g.,). The implicit assumption that human judgment is monolithic and characterized by consensus should be carefully examined in future work.

The AI tools used in this study were all general-purpose tools. Specialized tools, both open source and commercial, have been and are being developed to deal specifically with searching the literature and writing literature reviews (Asai et al., 2026; Zabaleta and Lin, 2026, e.g.,). This study has not compared our results to these specialized tools.

9 Conclusions

For better or worse, LLMs are now used to help write literature reviews. This study tested how well they do it, by following the process of writing a review step by step against a published review of the Spanish plume whose 102 articles are known. Five AI tools searched for the articles. Three were then given all of the 102 articles and asked to build a database, to write a synthesis, and to propose future work. Two human experts scored the syntheses without knowing which tool had written which.

The first question was whether the tools could find the relevant articles. The answer depended mainly on access. Without access to the university subscriptions, no tool found more than half of the articles; with access, the two coding agents found far more. But, a subscription was not the only obstacle. Some websites refused the tools’ automated requests even for articles that were free to read, and one database searched only titles, abstracts and keywords, so it missed articles that mentioned the term “Spanish plume” only in the body of the article. The tools also found recent articles more often than older ones.

The second question was whether the tools could interpret each article. The tools extracted text accurately; nearly all the quotations in their databases were word for word. They were less reliable at judging what an article did with the term “Spanish plume”. All three recognized the 18 articles that discussed the Spanish plume substantially, but they also classed many of the passing mentions as substantial. The tools often mistook a mention of the term for a discussion of it.

The third question was whether the tools could synthesize across articles. The syntheses were mostly accurate, but both reviewers found little insight in them. The tools overstated how far the literature agreed, and one repeated a description of the plume that appeared in several articles but conflicted with the original definition. Only an expert would have caught that. The tool that cited the most articles cited all the 18 substantial ones, but only by citing nearly everything. Thus, citing more did not necessarily mean choosing sources better. This result was likely constrained by the prompt, which was written to prevent errors and did not ask for interpretation.

The fourth question was whether the tools could propose future work. The tools did better here than might be expected. Two tools proposed research gaps close to those in the benchmark review, whereas the third repeated gaps the articles had already stated. Their weakness was focus, as all three tools proposed gaps on subjects broader than just the Spanish plume.

Taken together, these results show that the tools could help with a literature review but cannot be left to write one alone, at least at this time, results consistent with previous studies (Mostafapour et al., 2024; Schiller, 2024, e.g.,). They can find many of the relevant articles, extract what the articles say, and summarize it reasonably accurately, which potentially could save time. But their searches have gaps, their judgments of importance are unreliable, and they exhibit equally and often unwarranted confidence whether or not they are accurately reflecting the real state of the literature. Section 7 gives recommendations for using AI tools for constructing literature reviews, which come down to giving the tool the access it needs and checking its work at every step.

There are indications that different subjects may yield different results (Tang et al., 2025). Our conclusions in this study come from one topic, one run of each tool and two reviewers. The AI tools are changing quickly. Given the biases in finding literature in our study and others, access in the future may influence citation rates as more authors use and cite references provided by AI tools. The method can be repeated, however. Any published review with a known set of articles can serve as a benchmark, and repeating this test on other topics and with later versions of the tools would show how far these results hold.

Given our mixed results on the value of the AI tools, what is our take-away message? The tools can help with parts of a literature review, but they did not bring the insight that an experienced researcher brings to reading and synthesizing a literature. LLMs will no doubt continue to improve. But the process of researching the literature, reading it, reflecting on it, and writing up a synthesis of it remains essential to situating and motivating a research project. Outsourcing parts of the process of writing a literature review does a huge disservice to PhD students and researchers, who are expected to learn the context of their research. If they understand the literature no more deeply than the syntheses of the Spanish plume literature produced by these three tools, they will be unable to place their research within the breadth of the field, to any depth.

Acknowledgements.

The authors acknowledge financial support from the N8 Centre of Excellence in Computationally Intensive Research (N8 CIR). This work was carried out as a summer research internship at the University of Manchester and made use of the University’s Computational Shared Facility and the N8 computing facilities. Author CRediT statement: AAK: methodology, software, validation, investigation, data curation, writing – original draft, and visualization; DMS and AKAF: conceptualization, methodology, validation, writing – review & editing, supervision, and funding acquisition; ZZ: conceptualization, methodology, writing – review & editing, supervision, and funding acquisition; CM: conceptualization, methodology, writing – review & editing, and supervision.

Data availability statement.

The 102-article corpus can be found at Schultz et al. (2024). The public archive of the present research is available from Zenodo (https://doi.org/10.5281/zenodo.23849039) (Al Kalbani et al., 2026) and GitHub (https://github.com/abx40/spanish-plume-llm-literature-review). It is organized by experiment and contains the prompts, raw model outputs, anonymized per-theme reviewer ratings, cleaned per-paper outcomes and summary tables, the data and images for the figures that show results, and scripts that redraw the figures and recompute the reported numbers. The archive is released under a CC BY 4.0 licence. Publisher-hosted source-article PDFs, the benchmark’s full database, private authentication records, and named reviewer correspondence are not posted. The source PDFs and other non-public materials are held by the corresponding author; access is subject to source and reviewer permissions.

References

• Abdulhai et al. (2026) M. Abdulhai, I. White, Y. Wan, I. Qureshi, J. Z. Leibo, M. Kleiman-Weiner, and N. Jaques How LLMs distort our written language. arXiv preprint. Note: arXiv:2603.18161 External Links: Document Cited by: §5.1, §5.6.
• Agarwal et al. (2024) S. Agarwal, G. Sahu, A. Puri, I. H. Laradji, K. D. Dvijotham, J. Stanley, L. C. Mila, and C. Pal LitLLMs, LLMs for literature review: are we there yet?. External Links: Document Cited by: §1.
• Al Kalbani et al. (2026) A. Al Kalbani, D. M. Schultz, A. K. Anderson-Frey, Z. Zheng, and C. McGuire Data for “Can large language models review the scientific literature? A test against a 102-article benchmark”. Note: Zenodo External Links: Document Cited by: §8, §9.
• Asai et al. (2026) A. Asai, J. He, R. Shao, W. Shi, A. Singh, J. C. Chang, K. Lo, L. Soldaini, S. Feldman, M. D’Arcy, D. Wadden, M. Latzke, J. Sparks, J. D. Hwang, V. Kishore, M. Tian, P. Ji, S. Liu, H. Tong, B. Wu, Y. Xiong, L. Zettlemoyer, G. Neubig, D. S. Weld, D. Downey, W. Yih, P. W. Koh, and H. Hajishirzi Synthesizing scientific literature with retrieval-augmented language models. Nature 650, pp. 857–863. External Links: Document Cited by: §8.
• Carlson and Ludlam (1968) T. N. Carlson and F. H. Ludlam Conditions for the occurrence of severe local storms. Tellus 20, pp. 203–226. Cited by: §2.1, §5.3, §6.4.
• Ernst et al. (1993) E. Ernst, T. Saradeth, and K. L. Resch Drawbacks of peer review. Nature 363, pp. 296. External Links: Document Cited by: §8.
• Felizardo et al. (2025) K. R. Felizardo, A. Deizepe, D. Coutinho, G. Gomes, M. Meireles, M. Gerosa, and I. Steinmacher On the difficulties of conducting and replicating systematic literature reviews studies using LLMs in software engineering. IEEE/ACM International Workshop on Methodological Issues with Empirical Studies in Software Engineering (WSESE), pp. 20–23. External Links: Document Cited by: §5.5.
• Fery and Faranda (2024) L. Fery and D. Faranda Analysing 23 years of warm-season derechos in France: a climatology and investigation of synoptic and environmental changes. Wea. Climate Dyn. 5, pp. 439–461. External Links: Document Cited by: §5.3.
• Google (2026) Google Do your best research with NotebookLM. Note: The Keyword (Google blog), 8 June 2026, https://blog.google/innovation-and-ai/products/notebooklm/better-research-notebooklm/Accessed September 2026 Cited by: Table 1.
• Gray and Marshall (1998) M. E. B. Gray and C. Marshall Mesoscale convective systems over the UK, 1981–97. Weather 53, pp. 388–396. Cited by: §2.1, §3.4.
• Hauser (2025) A. S. Hauser The future of reviews. EMBO Reports 26, pp. 4397–4401. External Links: Document Cited by: §1.
• Jain et al. (2024) C. Jain, S. Vahdati, N. Gopan, I. F. Sbalzarini, and J. Lehmann Evaluating large language model literature reviews in interdisciplinary science: a systems biology perspective. In EKAW 2024: EKAW 2024 Workshops, Tutorials, Posters and Demos, 24th International Conference on Knowledge Engineering and Knowledge Management (EKAW 2024), External Links: Link Cited by: §3.1.
• Lewis and Gray (2010) M. W. Lewis and S. L. Gray Categorisation of synoptic environments associated with mesoscale convective systems over the UK. Atmos. Res. 97, pp. 194–213. Cited by: §2.1, §4.2.
• Lieberum et al. (2025) J. Lieberum, M. Toews, M. Metzendorf, F. Heilmeyer, W. Siemens, C. Haverkamp, D. Böhringer, J. J. Meerpohl, and A. Eisele-Metzger Large language models for conducting systematic reviews: on the rise, but not yet ready for use—a scoping review. Journal of Clinical Epidemiology 181, pp. 111746. External Links: Document Cited by: §1, §1.
• Morris (1986) R. M. Morris The Spanish plume—testing the forecaster’s nerve. Meteor. Mag. 115, pp. 349–357. Cited by: §2.1.
• Mostafapour et al. (2024) M. Mostafapour, J. H. Fortier, K. Pacheco, H. Murray, and G. Garber Evaluating literature reviews conducted by humans versus ChatGPT: comparative study. JMIR AI 3, pp. e56537. External Links: Document Cited by: §1, §9.
• Poe and Dryer (2026) M. Poe and D. B. Dryer AI is ruining scholarship. Note: The Chronicle of Higher EducationAccessed August 2026 External Links: Link Cited by: §1, §3.1, §3.3, §3.4.
• Scherbakov et al. (2025) D. Scherbakov, N. Hubig, V. Jansari, A. Bakumenko, and L. A. Lenert The emergence of large language models as tools in literature reviews: A large language model-assisted systematic review. J Am Med Inform Assoc 32, pp. 1071–1086. External Links: Document Cited by: §1.
• Schiller (2024) C. A. Schiller The human factor in detecting errors of large language models: a systematic literature review and future research directions. External Links: Document Cited by: §1, §9.
• Schultz et al. (2024) D. Schultz, D. Lowe, and F. Herrerias Azcue Spanish plume literature review and case studies. External Links: Link Cited by: §1, §2.2, §9.
• Schultz (2010) D. M. Schultz Are three heads better than two? How the number of reviewers and editor behavior affect the rejection rate. Scientometrics 84, pp. 277–292. External Links: Document Cited by: §8.
• Schultz et al. (2025) D. M. Schultz, M. V. Young, and D. J. Kirshbaum The Spanish plume elevated mixed layer: A review of its use and misuse within the scientific literature. Mon. Wea. Rev. 153, pp. 737–761. External Links: Document Cited by: §1, §2.1, §2.1, §2.2, Figure 2, §4.2, Figure 6, §6.4.
• Schultz (2005) D. M. Schultz A review of cold fronts with prefrontal troughs and wind shifts. Mon. Wea. Rev. 133, pp. 2449–2472. Cited by: §5.3.
• Tang et al. (2025) X. Tang, X. Duan, and Z. G. Cai Large language models for automated literature review: An evaluation of reference generation, abstract writing, and review composition. In Proc. 2025 Conf. on Empirical Methods in Natural Language Processing, pp. 1602–1617. External Links: Document Cited by: §1, §1, §5.1, §9.
• van Dort and Heuss (2026) I. van Dort and M. Heuss How do LLMs cite? A mechanistic interpretation of attribution in RAG. In Advances in Information Retrieval: 48th European Conf. on Information Retrieval (ECIR 2026), Lecture Notes in Computer Science, Vol. 16485, pp. 458–473. External Links: Document Cited by: §4.1.
• Wu et al. (2025) K. Wu, E. Wu, K. Wei, A. Zhang, A. Casasola, T. Nguyen, S. Riantawan, P. Shi, D. E. Ho, and J. Zou An automated framework for assessing how well LLMs cite relevant medical references. Nat. Commun. 16, pp. 3615. External Links: Document Cited by: §4.1.
• Wu et al. (2026) Y. Wu, X. Liu, Y. Feng, J. Ding, and X. Ma PaperAsk: A benchmark for reliability evaluation of LLMs in paper search and reading. In Proc. ACM Web Conference 2026 (WWW ’26), pp. 2330–2338. External Links: Document Cited by: §1, §1, §5.1, §5.6.
• Zabaleta and Lin (2026) M. Zabaleta and B. Lin SciLitBench: benchmark and design principles for LLM-powered systematic literature reviews. External Links: Document Cited by: §8.
• Zhang et al. (2026) T. Zhang, S. Fu, D. M. Schultz, and Z. Zheng Using large language models to produce literature reviews: usage patterns of microphysics parameterizations in WRF simulations from 2699 publications. Artificial Intelligence for the Earth Systems 5, pp. 250036. External Links: Document Cited by: §1, §4.1.
• Zhao et al. (2025) M. Zhao, F. Li, F. Cai, H. Chen, and Z. Li Can we trust LLMs to help us? An examination of the potential use of GPT-4 in generating quality literature reviews. Nankai Business Review International 16, pp. 128–142. External Links: Document Cited by: §1, §6.1.

注意转载授权 · 阅读原文