K-Dense 发布 Citation Telephone，把被引论断逐跳追溯到原始出处

In January 1980, the New England Journal of Medicine printed a five-sentence letter from Jane Porter and Hershel Jick. Among 11,882 hospitalized patients who had received at least one narcotic, they found only four well-documented cases of addiction in people with no history of it. The letter said nothing about patients taking opioids at home for months, yet it went on to be cited hundreds of times as evidence that addiction to prescribed opioids is rare. When Leung and colleagues traced its citations in 2017, they found 608 citing articles. Of those, 439 (72%) cited the letter as evidence that addiction was rare, and most never mentioned that its patients were in hospital.

Few of those authors set out to mislead anyone. Each cited the paper in front of them a little more confidently than the last, and the claim drifted one hop at a time, like a message in the children's game of telephone. The pattern is common. A meta-analysis of 28 studies of quotation accuracy in medical journals estimated that about a quarter of citations (25.4%) don't support the statement they're attached to, and about one in eight are major errors. A study of one citation network in biomedicine showed how a belief can gain authority through citation alone, including "the conversion of hypothesis into fact".

Checking a citation means opening the cited paper and finding the passage it was cited for. Checking where a claim really came from means doing that again for every paper it passes through. So today we're releasing Citation Telephone, a free tool that does the walking for you. Pick a cited sentence, and it follows the claim back through the papers it cites, reads each one, and shows you where the claim came from and what changed on the way.

One sentence, followed back

Here is a sentence from a 2024 paper in the Journal of Medicinal Chemistry: "More importantly, nuclear PD-L1 can promote cell proliferation in an immune-dependent manner without interacting with PD-1." It cites a 2024 review in Advanced Science. Citation Telephone opened the review's full text and found the passage it was cited for, which says nuclear PD-L1 "promotes immune-independent cell proliferation without binding to PD-1". That is the opposite of the citing sentence. The review in turn cites a 2022 study in Cancer Letters, which reports the finding from its own experiments: nuclear PD-L1 "promotes cell cycle progression in an immune-independent manner in BRAF V600E-mutated CRC."

The trail shows what a quick check would miss. The first hop is marked Changed, because one word ("dependent" for "independent") reverses the claim. The second hop has no label. The model's readings were split (partly matches 39%, matches 34%, goes further 25%), and none reached the bar for showing one. The split makes sense once you read both passages: the review states as a general property of nuclear PD-L1 what the original study showed in one kind of colorectal cancer. At the origin, the tool also compares your starting sentence with what the origin says, and that comparison comes back Changed (81%). You don't have to take the tool's word for any of this. Each step opens the passage it rests on, highlighted inside its paragraph, and the readings behind every label are one click away.

What each hop tells you

Each line on the trail compares the citing sentence with the passage the tool found in the cited paper:

• Matches: the paper says what it's cited for, allowing different wording and rounding.
• Partly matches: the paper backs part of what it's cited for, but not all of it.
• Goes further: the citing sentence claims more than the paper shows. It might be more certain, causal where the paper reports an association, or general where the paper studied one group.
• More cautious: the citing sentence claims less than the paper.
• Changed: the paper addresses the same question but says something else, such as a different number, population, cell type or direction.
• Not there: no passage of the paper states what it's cited for.

Every paper also gets a result, which decides whether the trail goes on. An origin reports the finding as its own result, from its own experiments, data or pooled analysis, and the trail ends there. A paper that passes it on states the claim as background and cites earlier work for it. The trace follows those references, and when a sentence cites several, it picks only the ones cited for this claim. A paper can also state the claim with no source given, or turn out not to say it at all, which is marked doesn't say it. If a paper's full text isn't open access, it is marked abstract only. That never counts as "doesn't say it", because a claim missing from an abstract may still be in the paper.

Two ways to start

The usual way in is through a paper. Paste the DOI, PubMed ID or PMC ID of any open-access paper in Europe PMC, or the arXiv ID of an arXiv paper under an open license such as CC BY, and Citation Telephone lists every sentence in it that cites something, grouped by section, with a filter to find the one you mean. Choose a sentence and the references it cites appear underneath; pick up to three to trace.

You can also start from a claim you found anywhere, such as a grant review, a news story or a slide. Paste the sentence and the DOI, PubMed ID or arXiv ID of the paper it cites, and the trace starts from that paper. Either way, the page fills in as each paper is read, and when the trail ends you can pick any reference that wasn't followed and trace that one too. A trail can also be downloaded, as a Markdown report or as JSON with every reading.

More trails

Machine learning results travel fast, and so do their citations. A 2026 review in Genes & Development states that "AlphaFold and related models now predict protein and protein-ligand complexes at near-experimental accuracy," citing both the 2021 AlphaFold 2 paper and the 2024 AlphaFold 3 paper. Citation Telephone reads both in full and marks each as an origin that partly matches. The AlphaFold 2 passage reports "accuracy competitive with experimental structures in a majority of cases" at CASP14, a result about single protein chains, not complexes or ligands. The AlphaFold 3 passage does cover protein-ligand complexes. It reports that the model "greatly outperforms classical docking tools such as Vina" without using structural inputs, which is a strong result, but not the one the review states. Each paper backs part of the sentence, and neither passage says that complexes are predicted at near-experimental accuracy.

Sometimes the cited paper says the opposite. A 2024 paper writes that "it is well-known that confidence intervals provide evidence of post hoc statistical power," citing a 2019 paper in General Psychiatry. That paper's own simulations show that post hoc power analyses "do not indicate true power for detecting statistical significance." The trail ends at an origin, and the hop is marked Changed.

Most trails are less dramatic, and a quiet trail is useful too. A 2024 paper says sleep is "a crucial period for brain memory consolidation," citing a 2022 systematic review of sleep loss and physical performance. The review mentions memory in one sentence that cites five papers for five different roles of sleep, and the trace follows only the one cited for memory, a 2019 meta-analysis that found a small to medium benefit of sleep on prospective memory. The trail runs two hops to an origin that reports its own pooled result. No hop goes further than its source or changes it, though the first is only a partial match, because the citing sentence says far more about sleep than the review does.

How it works

A trace repeats the same steps at every paper, and only two of them, finding the passage and judging it, need a model. Finding papers, reading their structure and following references are plain code.

1. Read the citations. Open-access papers are read from their Europe PMC full text, in the JATS XML format publishers deposit. In that format every in-text citation is linked to its entry in the reference list, so the tool knows exactly which reference each sentence cites, whether it is written as [12], as a superscript, as "(Smith et al., 2010)" or as a range like [3-6]. arXiv papers are read from the HTML that arXiv generates from their LaTeX, where citations are linked to the bibliography in the same way. For the rare paper with no citation markup at all, citations are matched in the text by number, or by first author and year.
2. Find the cited paper. Each reference is matched to its Europe PMC record by PubMed ID, PMC ID or DOI. Papers Europe PMC doesn't index, most of physics, chemistry, engineering and computer science among them, are found in OpenAlex instead. References without identifiers are matched by title, or through Crossref. A Crossref match only counts when the year agrees and either nearly every word of the title does too or, for a citation printed without a title, as chemistry and physics journals print them, the first author, volume and first page all agree.
3. Read it. A paper in the open-access subset is read in full, including tables and figure legends. So is an NIH author manuscript, from the article datasets PubMed Central publishes for text mining, and an arXiv paper, from its HTML. For any other paper, only the abstract can be read, and the tool says so.
4. Find the passage. The model reads the paper paragraph by paragraph, then sentence by sentence, and picks the passage closest to what the citing sentence cites it for. It is told how the sentence cites this paper ("[14]"), so that in a sentence citing several papers for different points, it looks only for this paper's part. It picks two candidates: the closest statement, and the paper's own result on the claim.
5. Judge it. For each candidate, the model gives a probability that it states what the paper is cited for, and compares it with the citing sentence. Separately, with the claim out of view, it decides whether the passage is the paper's own result or a description of earlier work. If the sentence cites other papers, it also scores which of them are cited for this claim.
6. Follow or stop. Fixed rules turn those readings into a result. Origins end a branch. Papers that pass the claim on are followed into the references cited for it, up to three per sentence, six hops and 24 papers per trace. Further down the trail, the tool also compares the claim you started with against each passage, so drift that builds up over several hops shows even when each single hop looks small.

The model is Jev, the System One decision model from TypeSafe that also powers Rigor Scan and Claimscape. Jev doesn't write prose. It answers typed questions with probabilities, which suits this tool for the same reasons it suits the other two. When it picks a passage, the only options it's offered are the paper's own paragraph and sentence IDs, so the evidence on screen is always text the paper contains. When it judges a passage, it returns probabilities, so the rule for "origin" or "goes further" can be written down and applied the same way to every paper. And TypeSafe quotes response times of 70 to 500 milliseconds, which is what lets a trail several hops long fill in while you watch. Every question, threshold and rule is published word for word under How a trace is made, and you can copy the whole method as JSON.

How we tested it

The numbers in this section are for method v0.2, the version the tool launched with. The tool now runs v0.3, and the update at the end of this section gives the same measurements for it.

Each hop rests on one judgment: given a sentence and a paper it cites, does the paper state what the sentence cites it for, and if so, where? We measured that judgment directly, on real citations.

The test sets

We built three sets of citation pairs from open-access papers in Europe PMC. For each citing paper, we picked sentences at random from its introduction or discussion, roughly 60 to 450 characters long, citing a single reference in set 1 and one to three in sets 2 and 3. For each sentence we chose one of the references it cites and looked it up in Europe PMC. Each citing paper contributed up to two pairs whose cited paper has open-access full text and, in sets 2 and 3, one whose cited paper is closed, so only its abstract can be read. Each also contributed a control: the same sentence paired with another open-access paper from its reference list that isn't cited anywhere in that paragraph. The controls are hard on purpose, because a paper from the same reference list is usually on the same topic.

The 257 pairs come from 79 citing papers on 44 topics, including cancer immunotherapy, the gut microbiome, dental caries, coral reefs and protein design. No citing paper appears in more than one set, so set 3 shares no sentences with the sets used to build and tune the tool.

All the labels were made by AI, not by human experts, so weigh the results with that in mind. Set 1 was labeled by a single annotator, an AI model working with us, after it had seen an earlier version's output, so it is not blind. Sets 2 and 3 were labeled by separate AI reviewers (instances of Claude, a different model family from Jev). They saw only the citing sentence, how it cites the paper, and the cited paper's text, never the tool's output. For each pair they gave a verdict (supports, partly supports, goes further, contradicts, not stated, or cannot tell from an abstract), quoted the best passage, and said whether that passage is the paper's own result and whether it cites other work. We dropped five pairs because a reviewer's run was interrupted. Three of them were on papers about pathogens and toxins, and set 3 avoided those topics.

What we measured

The found rate covers pairs where the tool read the cited paper's full text and the reviewer judged that the paper addresses what it is cited for (a verdict of supports, partly supports, goes further or contradicts). It is the share of those pairs where the tool also found a passage that addresses it. The false-find rate covers full-text pairs where the reviewer judged that the paper does not state what it is cited for, and it is the share where the tool reported a passage anyway.

We also counted how often a paper read only from its abstract was wrongly reported as not saying something, and how often the label on a hop (matches, partly matches, goes further, changed) and the result (own result, cites earlier work, no source given) agree with the reviewer.

Results

Jev's answers vary a little from run to run, so we ran the whole evaluation three times. The found counts moved by at most one pair per set (40 to 41 in set 1, 36 in every run of set 2, 38 in every run of set 3), and the false finds by at most one (2 to 3 in set 2, 0 to 1 in set 3). The table shows the first run. Set 2 shaped the final thresholds, so its numbers are optimistic. Set 3 gives the fairest estimate, and with 39 and 21 pairs its intervals are wide.

Closed papers behaved as designed. In all three runs, none of the 46 pairs whose cited paper could only be read from its abstract was reported as not saying something. Where the abstract states the finding as the paper's own result, the paper is still marked as an origin.

The label on each hop is the weakest part. Where the tool found support, its comparison agreed with the reviewer's in 47 to 55% of pairs, depending on the set and run. Nearly all the disagreements are the tool saying "partly matches" where the reviewer saw a full match, and we chose to err in that direction. "Goes further" and "changed" are shown only when the model gives them at least 50%. In every run of set 3, one citation that the reviewers judged sound got one of those warnings, and none of the three problems the reviewers found was labeled a match. Three problems are far too few to tell how reliably the tool catches them, so treat a warning as a reason to read the passage yourself. The tool's call on whether a passage is the paper's own result or background agreed with the reviewers in 68 to 81% of pairs.

The test sets also give a rough count of the problem itself. Of the 130 real citations the blind reviewers read, 12 (9%; 95% CI 5 to 15%) cited a paper that didn't state the claim or said something else. That is close to the rate of major quotation errors in the meta-analysis above.

What changed, and what didn't help

The first public version (v0.1) found the supporting passage in 31 of 42 pairs of set 2 (74%; 59 to 85%) the first time we ran it, before any tuning. Version 0.2 changes three things in the judging. Many misses had the right sentence highlighted but a support reading just under 0.5, so a passage now counts as support from 0.4. "Goes further" and "changed" are shown only at 50% or more, and below that the passage gets its best other reading, which cut warnings on sound citations across sets 2 and 3 from seven to one or two, depending on the run. And the definition of "matches" now says that parts of a sentence about other papers, or about the citing paper's own work, don't count against the passage.

It also changes three things in the reader. It treats any link into the reference list as a citation; one publisher marks citations differently, which had hidden every citation in one of the test papers. It reads citations from the text of papers that don't mark them up. And it keeps an author-year citation that opens a sentence with that sentence, rather than attaching it to the one before.

Three ideas didn't help, and we left them out. Giving the model the sentence before the claim, to resolve words like "these" or "such", removed one or two false finds per 45 controls but cost about five points of found rate. Asking the support question a second way and averaging the two answers added nothing beyond what the lower threshold already gave. A stricter definition of "partly matches" moved errors from "partly matches" into "goes further", which is worse for a reader.

Errors we still see

The misses cluster in a few kinds of sentence. Partial matches sit near the threshold: a sentence that cites a study of bats' lungs for a finding about their livers is labeled partial by the reviewer and was missed by the tool in two of the three runs. Citations of a resource rather than a finding, such as "We conducted experiments on the CXR dataset of the 2021 SIIM-FISABIO-RSNA Machine Learning COVID-19 Challenge [44]", get the right passage highlighted but a low support reading. Sentences that lean on the one before, such as "Only a few of them explored…", give the tool little to look for. The tool also misses subtle swaps of a single term. One citing paper turned "fibroblastic reticular cells" into "fibroblastic reticulocytes", and the hop came back as a match.

Robustness

Separately from accuracy, we ran the paper reader over 175 open-access papers from 133 journals, published between 2008 and 2026. None failed or stalled, and all 14,684 citation links in them resolved to a reference. We also ran 24 full traces end to end, several of them more than one hop deep, and all finished without an error.

Limits of this evaluation

The labels are AI judgments. The reviewers come from a different model family than the tool's model, which limits shared blind spots but doesn't remove them, and no human expert checked the labels. The samples are small, so the intervals above are wide, especially for set 3 and for the false-find rate. The selection is narrow too: every pair comes from an open-access paper in Europe PMC, mostly biomedical, and from an introduction or discussion sentence citing at most three references. Methods citations, longer citation lists and other fields may behave differently, and pairs about pathogens and toxins were dropped from labeling.

The labels also judge one hop at a time. Each extra hop on a trail is another chance for an error, so on a long trail, check each hop.

The labeled pairs and the tool's output from all three runs are available as a JSON file, so you can check any number on this page. The questions, thresholds and rules behind them are on the tool page under How a trace is made.

Update: method v0.3

Later on launch day we released method v0.3, which changes three things.

1. Citations a few sentences away are followed. When the sentence closest to the claim cites nothing itself, the tool now follows the nearest citing sentences before and after it anywhere in the paragraph, not only the sentences next to it. Reviews often state a finding over two or three sentences and cite it once at the end, and v0.2 reported those papers as giving no source.
2. More references are found. Papers Europe PMC doesn't index are looked up in OpenAlex. References printed without a title, as chemistry and physics journals print them, are matched through Crossref by first author, year, volume and first page. And where a journal groups several works under one number and cites them as 17a and 17b, each citation now goes to the right work. In 68 references without identifiers, drawn from 37 open-access papers, v0.2 found none and v0.3 finds 43, with an abstract to read for 27.
3. More papers are read in full. NIH author manuscripts are read from the article datasets PubMed Central publishes for text mining, and arXiv papers from their HTML, one request every three seconds so as not to load arXiv's servers. Abstracts missing from Europe PMC are taken from OpenAlex. For an author manuscript, or an arXiv paper that isn't under an open license, the tool shows only the sentence a result rests on.

We ran the evaluation three more times on the same pairs. Five cited papers in set 2 that v0.2 could only read as abstracts are now read in full from author manuscripts, so set 2 has 47 supporting pairs with full text instead of 42.

The table shows the first run. Across the three runs, the found counts moved by at most one pair per set (40 to 41 in set 1, 41 to 42 in set 2, 38 to 39 in set 3), and the false finds by at most one (0 to 1 in set 3). That is the same accuracy as v0.2, within the variation between runs. None of the 41 pairs read from abstracts was reported as not saying something, and the call on whether a passage is the paper's own result agreed with the reviewers in 69 to 74% of pairs.

The test sets have few sentences whose citation sits more than one sentence away, so they barely register the first change. On the case that prompted it, a 2024 review of PD-L1, v0.2 reported that the review gave no source, and v0.3 follows its citation two sentences on to the 2020 study the passage rests on. The sets also say little about the new sources outside biomedicine. Almost every cited paper in them is in Europe PMC and none has an arXiv copy, so the numbers above don't cover papers found through OpenAlex or read from arXiv, and a trail through physics, chemistry or computer science deserves a closer look at each hop. The pairs and all three v0.3 runs are in a second JSON file.

What it can't do

Citation Telephone can only read what is openly available. Full text comes from the Europe PMC open-access subset, from NIH author manuscripts in PubMed Central and from arXiv. Other papers that are free to read but not openly licensed, and everything behind a paywall, are read from their abstracts. Abstracts rarely say where a claim came from, so many trails end at a closed paper. Take the widely repeated claim that the gut holds ten times more bacterial cells than the body has human cells. Many papers cite a 2005 Science review for it, which can only be read as an abstract, while a 2016 re-estimate found the two numbers to be of the same order.

Books, reports, guidelines and websites can't be read at all, and neither can papers missing from both Europe PMC and OpenAlex. Claims made only in a figure can be missed, though tables and figure legends are read. The limits of three references per sentence, six hops and 24 papers keep a trace fast. Where a trail stops before six hops, you can continue it from that paper. Above all, the tool checks citations, not truth. An origin can itself be wrong, and a claim can be true and still badly cited.

Fix it with K-Dense Web

A broken trail usually leads to one of a few next steps: find a source that supports the claim as written, rewrite the sentence to match its sources, read the papers that were closed, or check the rest of a paper's citations the same way. The bar under every trail turns your choice into a brief for K-Dense Web, our AI co-scientist. K-Dense Web can retrieve full texts, search the wider literature and draft the corrected sentence with the right citations.

The brief lists the claim, every paper on the trail with its identifiers and result, and the passage each result rests on. It's built from fixed templates, so nothing in it is generated, and K-Dense Web fetches the papers itself.

Private by default

Your claim is sent to the model along with the cited papers' public text, and K-Dense doesn't store or log it. The trail lives in your browser tab. A link copied from the page reruns the trace from the same sentence, along with any papers you added to the trail by hand, so what you share is how to rebuild the trail, not a stored copy of the result. To keep the result itself, download it.

Try it

Citation Telephone is free and needs no account. The page opens on the nuclear PD-L1 trail, and the AlphaFold, post hoc power and sleep trails above are one click away. It's most useful on a sentence you're about to cite, a claim in a manuscript you're reviewing, or a number that sounds a little too clean. If it finds something interesting, or gets something wrong, we'd like to hear about it at contact@k-dense.ai.

注意转载授权 · 阅读原文