Who Decides What the AI Sees?​

I was honoured to be invited to keynote at the IGeLU (International Group of Ex Libris Users) Annual Conference 2026 in Taipei last month, and I dare say it went quite well. Colleagues online seem interested in the talk, so this write-up covers what I presented.

This is not a verbatim transcript of what I said at IGeLU, but it captures the substance of it plus further insights I have had since then.

Thanks for reading Aaron Tay's Musings about Librarianship! Subscribe for free to receive new posts and support my work.

A couple of days after my keynote, Lisa Hinchliffe published on Scholarly Kitchen - Who Keeps the Catalog of Connectors? The Coming MCP Discovery Problem. It is definitely worth reading as a comparison.

Thank you for inviting me to keynote at IGeLU 2026. I am honoured to be here. The opening keynote speaker yesterday mentioned that if you are over 55, you do not need to worry about his talk on Quantum AI. Unfortunately, what I am going to share today is directly relevant even if you are over 55, because it is happening right now.

My thesis is that ever since academic search moved online in the 2000s, we have lived through two distinct “discovery wars”. These corresponded with technological shifts towards federated search and later central discovery indexes.

My view is that academic search shifted once again in the 2020s. Today, academic search differs from past paradigms in three fundamental ways.

Firstly, Semantic and dense retrieval - We began moving away from pure Boolean and lexical matching towards semantic search methods grounded in dense retrieval and dense embeddings. An example is Web of Science Smart Search, which blends lexical matching with dense retrieval.

Secondly, direct synthesis via RAG - Rather than simply returning a ranked list of items, academic search engines began generating direct answers with citations to top retrieved records using retrieval-augmented generation (RAG). Primo Research Assistant is a prime example.

Thirdly, Agentic workflows - We are witnessing a transition from single-iteration, fixed-pipeline retrieval flows to iterative, agentic search.

The first shift largely altered search rankings and went unnoticed by most end users. With all due respect to the librarians who did notice these changes, it is arguably the least disruptive of the three.

The second shift was immediately and much more visible to everyone. As a result, Librarians have rightly focused on how direct answer generation alters search literacy, evaluation, and user behaviour.

However, it is the third and newest shift that I believe currently remains least understood within the library community. This transition towards agentic search coupled with direct answers will trigger what I call the “Third Discovery War”.

What Is Agentic Search?

Definitions of software agents have largely converged. Anthropic defines an agent as a system where LLMs dynamically direct and control their own processes and tool usage to achieve a specified goal1 . An even more concise definition: an agent is an LLM that uses tools in an iterative loop to achieve a goal.

By extension, agentic search occurs when an LLM is given access to search tools2, formulates a plan, and decides dynamically how to deploy those tools to satisfy an information need.

Agents typically run within an execution harness that provides deterministic functions to manage context windows, expose tools and skills, handle memory, and manage error recovery.

Not all AI-powered search engines or deep research tools are agentic systems. For example, Primo Research Assistant does not qualify as agentic search because its retrieval pipeline is fixed in advance. The system cannot decide on the fly to bypass a planned step or substitute tools.

A few months ago, I experimented with various tools installed in my AI environment via MCP (Model Context Protocol). My setup included the Wiley Scholar Gateway, SciSpace, scite, Undermind, Consensus, a vibe-coded Primo connector, and several others.

I noticed that, given the exact same prompt, the LLM would occasionally select scite, sometimes Consensus, occasionally both, but it consistently ignored other options like Primo Search, even when Primo was the most logical choice.

This prompted an obvious question: what actually governs these choices?

We have been here before… or have we?

Many of you in the audience are veterans of past discovery wars, having navigated the migration from federated search to central indexes, and you understand the underlying mechanics even more than I do. Even so, examining how agentic search via MCP and connectors mirrors and diverges from earlier architectures ensures we do not mistakenly fight the last war.

One constant remains: discovery still happens elsewhere. Lorcan Dempsey pointed this out back in 2007, reminding us that libraries must design for users who do not begin their research on library-managed interfaces.

The difference now is that, in addition to accommodating users arriving from web-scale or network level discovery platforms like Google Scholar or ResearchGate, we must also support users operating inside agentic environments. Some of these are modern web-scale platforms like ChatGPT and Claude; others are locally hosted, custom agent harnesses.

Is the library’s task easier or harder today?

In certain respects, it is easier. Protocols like MCP standardise how external agentic environments connect to content, eliminating the need to build bespoke integrations for every third-party service or simply hope a web crawler indexes your metadata repository.

In other respects, the task is considerably harder. In previous eras, regardless of where users initiated a search, link resolvers consistently routed them back to the library ecosystem to retrieve the full text. Today, agentic systems synthesise direct answers, meaning users frequently never navigate to the source content at all. This structural reality has understandably alarmed content owners since ChatGPT first launched.

It is important to notice that Agentic search introduces substantial technical complexity. Traditionally, librarians focused on two primary questions:

First, the index - Was our institution’s content properly indexed?

Secondly, the retrieval and ranking algorithms - Were the ranking algorithms neutral, or were they commercially biased?

Arguably, despite initiatives like the NISO Open Discovery Initiative (ODI), fighting for algorithmic transparency into ranking was always an uphill battle, so libraries primarily focused on index coverage.

Agentic search complicates this chain further. Imagine an agentic system executing a search to construct a synthesised response. Between the initial query and the final output lie multiple intermediaries, controlled by distinct commercial entities:

1. The host platform: The user may initiate work in a personal consumer account, an enterprise-licensed workspace, or a bespoke local client.

2. Installed tools and MCP servers: The environment must have access to specific connectors installed by the user or by administrators at the platform level3.

3. Tool selection: Even with tools installed, the model’s planner must decide to call that specific connector.

4. The retrieval pipeline: Once invoked, the tool’s proprietary retrieval algorithm queries whatever index it can access.

5. Contextual selection: Even if the ideal document is successfully retrieved and injected into the context window, the LLM may still omit it from the final synthesis or fail to cite it.

Federated Search 2.0?

Many of you were around during the era of federated search and you understand the idea. You configured connectors to disparate targets—full-text repositories, publisher platforms, and A&I databases—which were queried simultaneously in real time.

Users selected a collection bundle of connectors/sources, entered a search string, and the federated engine translated the query into target-specific syntax.

Users would then by greeted by loading progress bars as multiple sources responded. Most users found this deeply frustrating. Unlike users today who are more tolerant of longer reasoning latencies for deep research agents, users back then trained by Google Search expected instant results. To make matters worse, connectors were brittle and broke frequently.

The other problem was that relevance was not very good. Because search was conducted in real time (with results coming from different sources at different times), the metadata returned from various sources were different and not harmonised, making relevance ranking difficult. I remember some systems of the time did not even attempt relevance ranking, simply returning results in the order they arrived.

It was for reasons like this that federated search generally failed to work well for our users; by 2009 or the early 2010s, we had all begun moving to centralised index systems like Summon, EDS, and Primo Central.

You all understand how central indexes work; CDI by Ex Libris is a paradigm example. The discovery vendor would collect content from different sources in advance, then harmonise and pre-index the data. This resulted in a search experience that was fast and stable, with decent ranking—at least for the time4.

I am not going to pretend central indexes for academic search were perfect or free of issues. NISO’s Open Discovery Initiative was formed to address some of these problems. A persistent issue concerned the availability of rich metadata for aggregator collections; famously, the standoff between ProQuest and EBSCOhost was one such flashpoint. Another was the concern that discovery vendors’ algorithms were biased in favour of their own content.

But despite all these problems, I would argue that the central index era has been a genuine improvement, at least for undergraduates who no longer need to struggle to figure out which database to use. That remains true even though, more than a decade on, many librarians and researchers—particularly those outside STEM, given that central indexes tend to favour journal articles—still detest such systems.

When we look back at the era of central indexes, we remember the early uncertainty over whether content owners and publishers would be willing to contribute their content to be indexed in these systems.

Initially, many were hesitant. Eventually, however, academic libraries around the world adopted these systems as the primary access point on their homepages, and librarians began presenting evidence at conferences such as Charleston and ER&L demonstrating that failing to be indexed was equivalent to becoming invisible and unused. Faced with this reality, most academic publishers threw in the towel; today, with rare exceptions (e.g. law content providers), virtually all are indexed in CDI.

Using this logic, should we then assume content owners will be equally eager to provide access to agents via protocols like MCP?

The problem here, of course, is that the situation is no longer completely analogous. In the days of federated search and central indexes, having your content indexed—and hence discoverable—meant users were ultimately directed back to your platform. Whether they began on Google Scholar, Summon, or Primo did not matter; at the end of the day, was that not the entire goal: for your content to be discovered and accessed?

By 2026, however, this discovery bargain has broken down. More often than not, if your content is retrieved from an index, an AI will summarise it to generate a synthesised answer, leaving it far from certain that the user will ever click through to the primary source.

I find much of the scholarly publishing industry’s analysis of so-called “zero-click searches” unconvincing because the presentations I have seen typically rely on data assuming standard Google searches with AI Overviews reflect academic information retrieval behaviour. That said, this 2025 analysis of a quarter-million interactions across AI2’s tools—Paper Finder and ScholarQA—revealed that “84% of report threads had no recorded paper-link click”.

As already mentioned, the current crop of academic AI search tools is becoming increasingly agentic. Rather than running predetermined retrieval pipelines or fixed workflows, these systems are equipped with a set of tools and connectors; they are then expected to formulate a plan, deciding which tools to deploy and in what sequence to accomplish the task.

I have blogged before about how platforms like Undermind and Consensus have grown more agentic, dynamically choosing from their available toolsets to execute non-standard retrieval tasks e.g. such as identifying citations that a given paper could have cited, but omitted.

So is agentic retrieval simply Federated Search 2.0?

Yes and no.

Yes, in the sense that the conceptual model of agentic search—querying multiple sources via standardised protocols such as the Model Context Protocol (MCP) and modern APIs rather than relying entirely on a single pre-harvested central index—mirrors the federated search architectures of the 2000s. There have, of course, been substantial technological advancements since then.

My understanding is that federated search initially relied heavily on Z39.50, which proved notoriously brittle and slow in practice. The technology gradually shifted towards SRU and SRW, and eventually to REST APIs, which offered marked improvements; however, by that stage, the reputational damage was largely done and central discovery indexes had won out.

These days, MCP promises more robust, standardised connectors. Furthermore, combining modern hybrid retrievers e.g. coupling dense semantic retrieval with sparse lexical scoring—with large language models acting as relevance judges helps mitigate the poor result quality that doomed the original federated search era.

LLMs accessing sources via MCPs, and even more so via command-line scripts, are exceptionally fault tolerant. One of my first mind-blowing experiences using Claude Code was watching the model alter a Python script on the fly to correct an execution error; in old-school, non-agentic federated search, the system would simply fail and return a blank error screen.

As already mentioned, users today are also far more tolerant of slower response times. The normalised expectation of “deep research” modes and the routine experience of waiting while models reason, execute tool calls, and complete multi-step tasks have fundamentally reshaped user patience.

Beyond these improvements in technology and user expectations, agentic search is also fundamentally different in three key ways.

Firstly, whereas 2000s federated search required users or librarians to select manually the specific sources or connectors to query (such as pre-configured “Library best bets”), modern agents evaluate the information need and select the appropriate tools autonomously.

Secondly, unlike 2000s federated search, agentic search can iterate across multiple retrieval cycles, dynamically refining queries and adjusting which targets to explore based on intermediate outputs.

Lastly, as noted earlier, these days agentic search does not merely retrieve and rank the most relevant documents; it actively synthesises an answer from the retrieved literature.

The ability of agentic search systems to choose which sources to search and, in particular, to iterate the search is exceptionally powerful and should not be underestimated. In the wider information retrieval world, formidable benchmarks like BrowseComp Plus, which run extremely difficult needle-in-a-haystack evaluations, are quickly becoming saturated; an agent (such as GPT-5.5) equipped with a well tuned lexical retriever can achieve 83.1% answer accuracy.

There has been less testing in the academic search domain on the impact of agentic search, but I have found that even connecting a basic LLM to a vibe-coded Primo MCP can resolve common user difficulties, such as identifying the appropriate database to use.

As I blogged previously, many of our users struggle to find the right database, a challenge clearly demonstrated when reviewing Primo zero-result logs filtered to queries scoped to databases. I identified four primary issues stemming from user inexperience, but suffice it to say an LLM overcomes all of them.

One big reason users struggle to find the right database is that they have little to no sense of the appropriate level of granularity to query. For instance, a user looking for a consumer or marketing database to research the market for Rolex watches might naively search for “Rolex watches” scoped strictly to databases, leading to zero results.

Asking an LLM to search the library catalogue for databases on Rolex watches tends to overcome this failure mode. The model will run the initial query, retrieve zero results, and then reason that it should broaden the scope one level higher—querying terms like “consumer goods” or “luxury goods”—which successfully retrieves the relevant databases.

So what actually controls what the AI sees to generate the answer? Many factors and entities are involved.

The most obvious element influencing whether an MCP tool or connector is invoked is the text in its description field. This is used by the LLM/agent to decide whether to use a tool5.

Researchers have been asking me how to make their papers more discoverable to “AI search”—the natural evolution from ASEO (academic search engine optimisation) to GEO (generative engine optimisation). But now I wonder whether creators of these MCP tools will also begin competing to optimise their tool descriptions, ensuring their tool is favoured when the LLM decides which one to call.

The other major factor determining what is selected is the specific set of skills configured in your agentic environment. If you are unfamiliar with skills, you can think of them as pre-packaged prompts, instructions, or capability packages that the LLM can dynamically select.

These skills can provide instructions on how to run scripts, deploy MCP tools, and structure workflow steps. Just as with MCP tools, the LLM evaluates the skill’s name and description to decide whether to activate it.

Forward-looking librarians have suggested that we could configure bespoke skills and MCPs for researchers tailored to their disciplinary workflows.

In my slide example, I demonstrate a simple skill instructing the LLM to determine whether a query is STEM-related or humanities-related, and, based on that classification, query a distinct set of databases.

While this is a toy example, far more sophisticated skills already exist that orchestrate multiple MCPs, command-line tools, and database connectors to automate comprehensive literature reviews.

MCP tools and connectors are far from uniform; they expose distinct tool definitions with varying action schemas. From the perspective of discovery, however, the pivotal question is what gets indexed and searched, and what actually gets returned6.

The most common implementations grant access only to titles, bibliographic metadata, and abstracts. Some occasionally retrieve full text when a paper is open access.

Paywalled content, however, presents a different challenge7. Newer MCP servers, such as the Wiley AI Gateway, are explicitly designed to retrieve full-text chunks alongside comprehensive metadata from the complete paper.

Of course, even if a tool is set up and connected, the LLM may simply not select it.

Even if selected to execute a search, the tool might fail to retrieve the relevant record or rank it highly enough to pass it into the model’s context window.

Finally, even when the relevant item is retrieved and successfully grounds the generated answer, the model might still omit the citation.

Approaches to providing real-time access to your content to AI agent

Imagine you are a content owner who wants to provide real-time8 access to your content for AI agents and LLMs. What are your options?

Based on what we are seeing across the current landscape, there are four primary approaches.

The first obvious approach is to offer access as a standalone MCP service that users can install directly9. Scite, Consensus, and the Wiley AI or Scholar Gateway are prominent examples of this model10. We see similar strategies emerging from EBSCO, while several libraries e.g Yale, Northeastern University are doing the same for their discovery layers, catalogues, and special collections.

The simplest path for end-users is when a vendor registers its connector in the ecosystem equivalent of an OpenAI or Anthropic “app store”. Slightly more demanding is requiring users to configure a custom connector by pointing to a remote server URL. Finally, at the technical extreme, users can run MCP servers locally on their own machines.

Importantly, even if a content owner chooses not to maintain an official MCP server, external agents can still access that content if an underlying API exists. Users can either construct custom API connectors themselves or point their agents to unofficial, community-hosted remote MCP wrappers.

Even within this approach, critical architectural questions arise. At what level of aggregation should MCP tools operate? Should they be scoped at the individual database level (such as PsycINFO on EBSCOhost)? At the broader platform level (querying all databases licensed across a provider)? Or organised into curated subject bundles—reminiscent of the classic “library best bets”?

It is also important to note that the institution itself may have a say in which tools are available by exercising control at the institutional or enterprise level.

Institutions that subscribe to enterprise agreements—such as ChatGPT Edu, Claude for Education, or institutional Gemini plans—often have the ability to push and prioritise specific MCP servers to their users via central administrative controls. Furthermore, several MCP servers are emerging that are architected exclusively for enterprise deployments and cannot be installed individually on a user’s personal account. Notable examples include enterprise MCP offerings from ThirdIron Libkey and Clarivate’s Nexus Connect.

At my own institution, for instance, we maintain our own “SMU AI Studio” interface (built on LibreChat), where specific MCP tools and agents are configured by default and we are currently working to add Primo via Nexus Connect.

The problem with relying on individual MCPs is that, without substantial aggregation, users will be left having to figure out which tools to select and how to configure them all. Can you imagine having to install 20 to 30 separate MCPs simply to gain comprehensive access to your institution’s subscribed collections?

This is precisely where Clarivate’s Nexus Connect enters the picture. It represents an ambitious bid to reposition the academic library as the institutional gateway through which AI agents access scholarly literature and university services.

Crucially, it is not merely a Primo and Alma MCP server. While Primo and the Central Discovery Index (CDI) underpin its initial discovery capabilities, the longer-term vision is substantially broader. Clarivate explicitly characterises Nexus Connect as:

“a single, university-branded integration layer”

The core premise is that universities should not have to depend on students and researchers individually discovering, evaluating, and activating disparate MCP connectors for every separate database, publisher platform, or library system. Instead, the institution provides a single, centrally managed integration point into its AI environment—complete with entitlement-aware access controls. In short, users just install ONE university branded MCP connector.

While I am on the Clarivate Academia AI Advisory Council, I don’t have much insight into how this is going to work.

A couple of questions off the top of my head

1. Will third party content owners and vendors want to be aggregated this way?

2. How much flexibility and transparency will institutions have in determining how this institutional gateway works?

Another potential approach is content owners partnering directly with AI labs and foundational platforms. For example, both EBSCO and Wiley appear to have struck agreements with Perplexity11, allowing institutions that subscribe to both Perplexity and either content provider to query these “premium sources” automatically behind the scenes, bypassing the need for an MCP connector entirely12.

To date, Perplexity is the most prominent platform pursuing such publisher deals; however, if OpenAI or Anthropic were to follow suit, the discovery landscape could shift dramatically.

The final method is included for the sake of completeness. Details on Elsevier’s LeapSpace are scarce, but my understanding is that it represents, in several respects, a return to the era of central discovery indexes. Elsevier appears to be asking other content owners to deposit the full text of their publications into LeapSpace to sit alongside its own Scopus and ScienceDirect corpus13. LeapSpace then operates as a closed, unified platform executing agentic workflows such as discovery, synthesis, and report generation.

This looks to me to mirror the central index architecture of Summon, Primo Central, and EDS, re-engineered behind an agentic research interface. If successful, this approach will naturally offer distinct operational advantages over real-time querying across distributed endpoints—though, once again, relevance ranking will need to be exceptional to navigate such colossal data volumes. The critical question remains whether content owners will be willing to trust Elsevier enough to deposit their full-text assets in the age when content owners worry about data being trained on14.

This is the exact dilemma that Summon, EDS, and Primo Central confronted in the 2010s; the key difference today is that, so far, only a single commercial initiative appears to be pursuing this path15.

If you find this kind of independent analysis of AI-powered academic discovery useful, consider buying me a coffee. I write these pieces in my own time, and your support helps keep them coming.

Buy me coffee via ko-fi!

More and better user analytics due to AI agents?

With so many moving parts determining what the AI sees, one might assume it is impossible to gain any clear picture of what is actually happening.

Interestingly enough, this may not remain the case, as content owners have begun demanding that AI agents provide far better analytics.

A very recent example is SPUR (Standards for Publisher Usage Rights), a consortium comprising virtually all the heavyweights of the news industry, including the BBC, the Associated Press (AP), and several others. Its newly published Content Telemetry Standard proposes a common framework through which AI agents and automated systems can record and report five distinct event types:

Its a very recently published Content Telemetry Standard proposes a common framework through which AI agents and other systems can record and report five types of events:

a) Retrieved – Content is fetched from a source.

b) Grounded – Content enters the LLM’s generation context, where it can potentially influence the answer.

c) Cited – Content is explicitly referenced in a generated response.

d) Presented – Content or a source reference is displayed or otherwise made accessible to a recipient.

e) Engaged – A user or agent interacts with the presented content or reference, such as clicking a citation, expanding a preview or copying text.

Importantly, the standard distinguishes between content that is merely retrieved, content that actually enters an LLM’s context, and content that is cited or presented. This potentially allows publishers to measure usage that conventional website analytics cannot capture.

However, this is a proposed reporting framework, that will need support from the AI labs.

You might be wondering why we should care about standards for the news industry. The reason is that COUNTER which is very much about usage standards for the academic publishing industry is currently in phase 2 of updates is planning to reference SPUR Content Telemetry

As of time of this writing, the COUNTER - Best Practice on Generative and Agentic AI usage metrics phase 2 is out.

Why would COUNTER do that? Because as substantial as academic publishing may seem, in the grand scheme of things it remains a relatively modest market in the grand scheme of things. That is why it is essential to align with standards established by the far larger and more influential news industry.

Even more recent is the launch of the Trusted Retrieval and Attribution for Content Ecosystems initiative at the Frankfurt Book Fair. It currently encompasses three interconnected projects:

TRACE: Trusted Retrieval & Attribution for Content Ecosystems is an initiative aimed at bringing together members of the scholarly publishing and AI communities to benefit everyone in the research communications ecosystem. TRACE aims to accelerate the development of the technical infrastructure needed to improve the trustworthiness of AI systems in scholarly communication. It does so by supporting the development and adoption of content exchange standards, provenance tracking mechanisms, and usage reporting protocols.

At one end is COUNTER’s work to update usage reporting for AI agent searches. At the front end is a project led by STM and Ithaka S+R focused on understanding how AI systems and agents discover and access scholarly literature. In the middle sits a NISO project developing a minimum metadata model for provenance. The core premise is that to track content in line with SPUR recommendations, every content asset must be accompanied by a standardised baseline of metadata.

I think it is fair to say that content owners are the primary drivers behind the projects and initiatives mentioned earlier, and their focus centres predominantly on “content-usage accountability” enabling them to demonstrate the value of their assets and charge accordingly.

However, as librarians and researchers, we also demand “retrieval accountability”. We care not only that the correct content was credited for influencing the generated answer, but also whether the retrieval process itself was executed rigorously: what sources were actually searched, and how effectively the retrieval was performed.

The classic method for evaluating retrieval is to benchmark against a gold-standard set of relevant papers, testing whether the system can retrieve most of these known documents at a given cut-off, typically measured via Recall@k.

Yet even when we establish that relevant papers were missed, do we actually understand why?

Librarians and organisations like the NISO ODI (Open Discovery Initiative) have long fought for transparency in search.

While librarians have worried about algorithmic bias for years, measuring it proved exceptionally difficult due to the lack of an objective, neutral baseline against which to compare results.

Although bias can be evaluated through multiple lenses, one viable approach is to quantify it as outcome disparity.

Suppose you have verified that there are 20 relevant articles in ProQuest and 10 relevant articles on the EBSCOhost platform16. Within the top 50 retrieved results, however, you find 16 relevant papers from ProQuest and only 4 from EBSCOhost.

Arguably, this demonstrates that ProQuest content is over-represented: it achieves a recall of 0.8 (16 out of 20), which is double the recall for EBSCOhost content at 0.4 (4 out of 10), suggesting algorithmic favouritism towards ProQuest17.

To conclude, what should libraries ask for? I would advocate for direct access to execution traces, substantially more transparent documentation, and audit-friendly interface features. Furthermore, librarians must take the initiative to design and run their own independent evaluations.

So to answer the question with which I began: what determines what the AI sees?

The answer is a complex constellation of factors:

• The specific AI platform and/or foundation model chosen by the user.

• The administrative configurations and governance controls applied within institutional or enterprise AI systems.

• The explicit tool definitions, parameters, and descriptions configured within each connector or MCP server.

• The internal mechanics of the retriever and the orchestration of the broader retrieval pipeline.

In the first two discovery wars, librarians fought to ensure that their collections were indexed and accessible. In the third discovery war, that is no longer sufficient. We must also ask whether AI agents are choosing to search those collections, whether relevant material is actually being retrieved, and whether it ultimately influences the answers users receive. The challenge is no longer simply making library content available to AI systems, but ensuring that the discovery process itself remains accountable. Libraries have spent decades advocating for transparency in discovery. The third discovery war gives us every reason to continue that fight.

Thanks for reading! If you enjoyed this deep dive into the changing academic discovery landscape and would like to support more independent writing on AI and libraries, you can buy me a coffee. Every contribution is appreciated.

1

For this talk, I am going to use “tools”, “connectors” and “MCP” interchangably. Technically speaking, tools are the capabilities the agents can invoke (e.g. search, code execution), connectors are integrations that allow agents to access external systems and services, including their data and capabilities. MCP (Model Context Protocol) is a standard protocol for exposing tools and other capabilities to agents. MCP servers are one way of implementing such integrations.

2

Practical search agents also require complementary non-search utilities, including document parsers, citation extraction tools, and persistent scratchpads.

3

See footnote #1

4

Of course, veterans of this “war” know this isn’t quite true. We struggled with issues like balancing the retrieval and ranking of total different item types with varying amount of metadata/full text as articles indexed with full-text crowded out everything else. The irriating problem of book reviews always seemingly ranked higher than the book itself and in general known item search was problematic. See my blog posts from 2011-2015 for a historical lookback.

5

Of course the LLM is heavily influenced by it’s pretraining and post-training. In general, a more well known and widely known tool will be known to the LLM even without the “reading” the description field.

6

The two things might not be the same. For example, a discovery service like Consensus MCP might have a partnership to do retrieval over full-text of a publisher like Wiley. However, it might only be able to feed the title or abstract to the LLM.

7

One important complication I have largely glossed over is authentication and authorisation. Traditionally, library discovery systems could search an index without establishing whether users were entitled to the full text, with authentication typically occurring after users clicked through to the publisher. With agentic search, however, the AI may need to retrieve and process the full text before generating an answer. How does an agent operating within a researcher’s personal ChatGPT or Claude account establish their institutional entitlements? And even when access is authorised, do the relevant licences permit AI processing? Different MCP and institutional gateway architectures will need to address these issues.

8

As opposed to provide content for models to train on.

9

It isn’t as clearcut as one publisher, one MCP. Silver Chair’s Research Bridge appears to one MCP connector for an aggregation of publishers (particularly for smaller publisher?), while Wiley has similar ambitions.

10

There is probably a distinction to be made between discovery services like Consensus, Scite offering access via MCP and actual publishers like Wiley making the same move. In general, the former group are not offering additional content beyond what is open (though some may have special publisher partnerships) and benefit purely from being used to search. The latter group will need a business model for the use of their content.

11

Others include Statista, Pitchbook, BMJ.

12

EBSCO’s partnership integrates EBSCOhost journal content into Perplexity Premium Sources while EBSCO's partnership integrates EBSCOhost journal content into Perplexity Premium Sources.

13

I was under the mistaken impression until recently that Elsevier does not provide their content via MCP. But they in fact do. That said, this looks extremely pricey, catering to enterprise R&D companies and not Universities.

14

Elsevier now explicitly confirms licensing agreements for selected full-text indexing from BMJ Group, IOP Publishing, Sage, NEJM Group, JAMA Network and several others.

15

Arguably Scite and Consensus are going around trying to sign publisher agreements are trying the same strategy as leapspace, but I am unsure about the degree of intergration they get from their publisher agreements. For example, Wiley shared with us Consensus agreement with them is just for discovery/matching purpose but the full text cannot be used to generate the answer.

16

Let’s pretend for the moment there are no overlapping articles on both platforms!

17

In reality, of course this is not iron clad. Traditionally such results can be explained by differences in metadata quality e.g. on EDS, certain A&I records have richer metadata and hence are more likely to be retrieved via the same record in Summon.

注意转载授权 · 阅读原文