Grobid 如何用 Crossref 元数据丰富文献书目信息

 5 分钟阅读。

 5 minute read.

Crossref 成员每天注册约 37,000 条记录。所有这些元数据都通过我们的免费 API 与全球学术社区公开共享。我们每月收到超过 20 亿次 API 调用，这表明它被全球数百种工具和服务使用，支持学术元数据的发现与评估。作为我们博客新系列的一部分，我们将聚焦于依赖 Crossref 元数据服务学术社区的各类工具和集成。在本篇开篇文章中，我们与 Grobid1 的开发者兼维护者 Luca Foppiano 进行了交流，了解 Grobid 如何利用 Crossref 元数据来改进从学术 PDF 中提取和丰富书目信息的工作。

Crossref members register around 37,000 records daily. All this metadata is openly shared with the global scholarly community through our free APIs. We receive over two billion calls to our API each month, signalling its use by hundreds of tools and services worldwide, supporting the discovery and assessment of scholarly metadata. As part of a new series in our blog, we will spotlight the diversity of tools and integrations that rely on Crossref metadata to serve the scholarly community. In this initial post, we chat to Luca Foppiano, the developer and maintainer of Grobid1, to learn about how Grobid uses Crossref metadata to improve the extraction and enrichment of bibliographic information from scholarly PDFs.

如果你曾尝试从 PDF 中提取书目数据，就会知道这并不总是件简单的事。手动复制作者姓名、标题和参考文献既繁琐又枯燥。版式中常常包含不可见字符，需要有人清理；即便是自动提取也常常失败，因为 PDF 格式是为跨平台阅读文档而设计的，并非为了干净、结构化的元数据。而如果你打算批量处理大量文件，问题就会呈指数级增长。这正是 Grobid 旨在解决的核心问题。

If you’ve ever tried to extract bibliographic data from a PDF, you know this is not always straightforward. Copying author names, titles, and references by hand is tedious. Often, the layout includes invisible characters that someone has to clean, and even automated extraction often fails because the PDF format was conceived for reading documents across platforms; not for clean, structured metadata. Then, if you are considering doing this in bulk, with long lists of files, the problem grows exponentially. This is the core problem that Grobid was conceived to solve.

走进 Grobid

Enter Grobid

Grobid 是一个开源软件库，由 Patrice Lopez 编写，帮助用户从 PDF 文件中提取和解析技术与学术文本。这意味着你可以将一篇学术 PDF 作为输入，Grobid 会提取结构化的书目元数据，包括标题、摘要、作者、所属机构、关键词、参考文献等。与 Crossref 元数据的集成使 Grobid 能够进一步提升这些元数据的质量，并提高整个流程的整体准确性。

Grobid is an open-source software library, written by Patrice Lopez, that helps users extract and parse technical and academic text from PDF files. This means you can use a scholarly PDF as input, and Grobid will extract structured bibliographic metadata, including title, abstract, authors, affiliations, keywords, references, and more. Integration with Crossref’s metadata allows Grobid to further augment the quality of that metadata and improve the overall accuracy of the process.

Grobid 被 ResearchGate、Academia.edu、HAL Research Archive、欧洲专利局、法国国家科学研究中心科学技术信息研究所（INIST-CNRS）、Mendeley、CERN（Invenio）、Internet Archive¹ 等集成服务以及研究项目²所使用。

Grobid is used by integration services like ResearchGate, Academia.edu, HAL Research Archive, the European Patent Office, The Institute of Scientific and Technical Information - French National Centre for Scientific Research (INIST-CNRS), Mendeley, CERN (Invenio), Internet Archive¹, and research projects².

Grobid 如何使用 Crossref 系统？

How does Grobid use Crossref systems?

在 2026 年第一季度，我们的 REST API 收到了来自 Grobid 软件客户端超过 1 亿次请求。Grobid 查询 REST API 并检索完整参考文献的元数据，补充缺失的元素并提高整个流程的整体准确性。此外，这一集成充分利用了 Crossref 的不同 API 池2，允许用户发出礼貌请求，或在其已是 Metadata Plus 订阅者的情况下在软件中包含其 Metadata Plus API 密钥³。

During the first quarter of 2026, our REST API received more than 100 million requests from Grobid software clients. Grobid queries the REST API and retrieves the metadata for the full reference, adding missing elements and improving the overall accuracy of the process. Additionally, this integration makes full use of Crossref’s different API pools2 by letting users make polite requests or include their Metadata Plus API keys in the software if they are already Metadata Plus subscribers³.

例如，出现在学术文档末尾的一条非结构化参考文献可能如下所示：

For example, an unstructured reference that appears at the end of a scholarly document might look as follows:

Rada, F, et al. “Osmotic and turgor relations of three mangrove ecosystem species.” Australian Journal of Plant Physiology, vol. 16, no. 6, 1 Dec. 1989, pp. 477–486, https://doi.org/10.1071/pp9890477.

Rada, F, et al. “Osmotic and turgor relations of three mangrove ecosystem species.” Australian Journal of Plant Physiology, vol. 16, no. 6, 1 Dec. 1989, pp. 477–486, https://doi.org/10.1071/pp9890477.

如我们所见，这种特定的引用风格只包含第一作者。其他风格可能会使用期刊缩写名，甚至完全省略期刊名。经过 Grobid 处理后，这条参考文献会被重新组织为一种名为 Text Encoding Initiative（TEI）的特定标准格式。如果它看起来复杂，不必惊慌，这些格式并非供人阅读，而是供机器以及软件之间的交互使用。我们想展示的是描述参考文献的两行内容是如何被标记和结构化的（在这方面，拥有包含丰富元数据的 DOI 无疑大有帮助）：

As we can see, this specific citation style includes only the leading author. Other styles might, for example, use the journal abbreviated name or even skip it entirely. After being processed by Grobid, this reference is reorganized into a specific standard format called Text Encoding Initiative or TEI. Don’t be alarmed if it looks complex, these formats aren’t intended for humans to read, they’re intended for machines and interactions between software. What we want to show is how two lines describing references are tagged and structured (for which having a DOI with rich metadata definitely helps):

<biblStruct status="consolidated" source="crossref" xml:id="b1">
                        <analytic>
                            <title level="a" type="main">Osmotic and Turgor Relations of Three Mangrove Ecosystem Species</title>
                            <author>
                                <persName>
                                    <forename type="first">F</forename>
                                    <surname>Rada</surname>
                                </persName>
                            </author>
                            <author>
                                <persName>
                                    <forename type="first">G</forename>
                                    <surname>Goldstein</surname>
                                </persName>
                            </author>
                            <author>
                                <persName>
                                    <forename type="first">A</forename>
                                    <surname>Orozco</surname>
                                </persName>
                            </author>
                            <author>
                                <persName>
                                    <forename type="first">M</forename>
                                    <surname>Montilla</surname>
                                </persName>
                            </author>
                            <author>
                                <persName>
                                    <forename type="first">O</forename>
                                    <surname>Zabala</surname>
                                </persName>
                            </author>
                            <author>
                                <persName>
                                    <forename type="first">A</forename>
                                    <surname>Azocar</surname>
                                </persName>
                            </author>
                            <idno type="DOI">10.1071/pp9890477</idno>
                        </analytic>
                        <monogr>
                            <title level="j">Australian Journal of Plant Physiology</title>
                            <idno type="ISSN">0310-7841</idno>
                            <idno type="ISSNe">1446-5655</idno>
                            <imprint>
                                <biblScope unit="volume">16</biblScope>
                                <biblScope unit="issue">6</biblScope>
                                <biblScope unit="page" from="477" to="486"/>
                                <date type="published" when="1989-12-01"/>
                                <publisher>CSIRO Publishing</publisher>
                            </imprint>
                        </monogr>
                        <note type="raw_reference">Rada, F., Goldstein, G., Orozco, A., Montilla, M., Zabala,O. &amp; Azócar, A. Relaciones Osmóticas</note>
                    </biblStruct>

<biblStruct status="consolidated" source="crossref" xml:id="b1">
                        <analytic>
                            <title level="a" type="main">Osmotic and Turgor Relations of Three Mangrove Ecosystem Species</title>
                            <author>
                                <persName>
                                    <forename type="first">F</forename>
                                    <surname>Rada</surname>
                                </persName>
                            </author>
                            <author>
                                <persName>
                                    <forename type="first">G</forename>
                                    <surname>Goldstein</surname>
                                </persName>
                            </author>
                            <author>
                                <persName>
                                    <forename type="first">A</forename>
                                    <surname>Orozco</surname>
                                </persName>
                            </author>
                            <author>
                                <persName>
                                    <forename type="first">M</forename>
                                    <surname>Montilla</surname>
                                </persName>
                            </author>
                            <author>
                                <persName>
                                    <forename type="first">O</forename>
                                    <surname>Zabala</surname>
                                </persName>
                            </author>
                            <author>
                                <persName>
                                    <forename type="first">A</forename>
                                    <surname>Azocar</surname>
                                </persName>
                            </author>
                            <idno type="DOI">10.1071/pp9890477</idno>
                        </analytic>
                        <monogr>
                            <title level="j">Australian Journal of Plant Physiology</title>
                            <idno type="ISSN">0310-7841</idno>
                            <idno type="ISSNe">1446-5655</idno>
                            <imprint>
                                <biblScope unit="volume">16</biblScope>
                                <biblScope unit="issue">6</biblScope>
                                <biblScope unit="page" from="477" to="486"/>
                                <date type="published" when="1989-12-01"/>
                                <publisher>CSIRO Publishing</publisher>
                            </imprint>
                        </monogr>
                        <note type="raw_reference">Rada, F., Goldstein, G., Orozco, A., Montilla, M., Zabala,O. &amp; Azócar, A. Relaciones Osmóticas</note>
                    </biblStruct>

此外，还有另一个工具，通过 Crossref 批量元数据选项来补充 API 查询。Grobid 背后的团队还开发了 Biblio-glutton⁴，这是一个“专用于科学书目信息”的实用工具，可作为 Crossref（及其他）学术元数据的本地高性能缓存。本地优先意味着你可以将其与公共数据文件⁵一起使用，部署自己的 Crossref 数据库副本，按需进行更新，并根据需要任意次查询。

In addition, there is another tool that complements API queries with Crossref bulk metadata options. The team behind Grobid also developed Biblio-glutton⁴, a utility “dedicated to scientific bibliographic information” that serves as a local, high-performance cache of Crossref (and other) scholarly metadata. Being local first means you can use it with the public data file⁵, deploy your own copy of the Crossref database, make updates as needed, and query as many times as you require.

Grobid 最近改进了与 Crossref 的集成，以利用 REST API 礼貌池⁶，并新增了更多元数据声明，例如用户所要求的 CRediT 分类法和利益冲突声明。

Grobid has recently improved its integration with Crossref to make use of the REST API polite pool⁶ and added more metadata statements, such as the CRediT taxonomy and conflict-of-interest statements, which were requested by users.

这种 Crossref 集成的另一个协同之处在于，用户的输出可以充实以资助元数据，识别数据集（数据引用端点）和软件的提及，并通过使用模块，纳入其他学科特定元素，如天文实体、超导材料以及以 Wikidata ID 标识的实体。

Another synergistic aspect of this Crossref integration is that users’ outputs can be enriched with funding metadata, identify mentions of datasets (data citation endpoint) and software, and through the use of modules, including other discipline-specific elements like astronomical entities, superconductor materials, and entities identified with Wikidata IDs.

你是否将 Grobid 与 Crossref API 一起使用过？我们很想在评论中听听你的经验或用例。

Have you used Grobid with the Crossref API? We’d love to hear about your experience or use cases in the comments.

Crossref 拥有来自全球超过 25,000 个成员组织贡献的逾 1.8 亿条记录。我们通过 REST API 公开提供所有元数据。如果你的工具也能从集成我们的开放元数据中受益，你还可以查阅我们的文档并测试不同的元数据检索选项。

Crossref holds over 180 mln records contributed by more that 25,000 member organisations from across the world. We make all the metadata openly available via our REST API. If your tool can also benefit from integrating our open metadata, you can also explore our documentation and test different metadata retrieval options.

参考文献

References

1. Houten, S., Hendricks, G., & Polischuk, P. (2024). Rebalancing our REST API traffic. https://doi.org/10.64000/pq6bm-0qs92 ↩︎

2. Besançon, L., Cabanac, G., Labbé, C., Magazinov, A., di Scala, J., Tkaczyk, D., & Weber-Boer, K. (2025). Detection of metadata manipulations: Finding sneaked references in the scholarly literature (Version 1). arXiv. https://doi.org/10.48550/ARXIV.2501.03771 ↩︎

3. Grobid 文档。 https://grobid.readthedocs.io/ ↩︎

4. Biblio-glutton。 https://github.com/kermitt2/biblio-glutton ↩︎

5. Rittman, M., del Ojo Elías, C., & Montilla, L. (2026). 2026 public data file now available. https://doi.org/10.64000/7s70g-drz77 ↩︎

6. Rittman, M., & Montilla, L. (2025). Announcing changes to REST API rate limits. Crossref. https://doi.org/10.64000/wadve-3tj60 ↩︎

1. Houten, S., Hendricks, G., & Polischuk, P. (2024). Rebalancing our REST API traffic. https://doi.org/10.64000/pq6bm-0qs92 ↩︎

2. Besançon, L., Cabanac, G., Labbé, C., Magazinov, A., di Scala, J., Tkaczyk, D., & Weber-Boer, K. (2025). Detection of metadata manipulations: Finding sneaked references in the scholarly literature (Version 1). arXiv. https://doi.org/10.48550/ARXIV.2501.03771 ↩︎

3. Grobid documentation. https://grobid.readthedocs.io/ ↩︎

4. Biblio-glutton. https://github.com/kermitt2/biblio-glutton ↩︎

5. Rittman, M., del Ojo Elías, C., & Montilla, L. (2026). 2026 public data file now available. https://doi.org/10.64000/7s70g-drz77 ↩︎

6. Rittman, M., & Montilla, L. (2025). Announcing changes to REST API rate limits. Crossref. https://doi.org/10.64000/wadve-3tj60 ↩︎

注意转载授权 · 阅读原文。中文正文为 AI 翻译，请以原文为准。

注意转载授权 · 阅读原文