We benchmarked four ways to give AI agents your files. Here is what actually matters.

March 31, 2026

AI agents need access to your files. Your Slack messages, Google Drive docs, GitHub repos. The question is how you connect them.

We tested four approaches. Same AI model. Same files. Same questions. The only difference: how the agent connects to the data.

Retrieval is a programming problem, not a search problem. ctxd gives agents a query language they write like code. It returns structured results they can reason over. It indexes every file across every connected app. That is why ctxd produced 4.8x more evidence-backed facts than the next best alternative.

E2E: 100 cross-source questions over a synthetic Series A startup (91 Drive docs, 12 Slack channels, 441 GitHub files, 3 months of activity). Each answer verified by an AI judge. Search: 100 queries across 523 highly similar papers from the same topic with deliberate vocabulary mismatch. Same AI model (GPT-5) across all backends. ctxd leads on every metric.

The problem

When an AI agent needs to answer a question about your company, it has to search your apps. But your apps do not share a search bar. Drive has one API. Slack has another. GitHub has a third. Each with its own query format, authentication, and quirks.

There are several ways to solve this. They differ in two dimensions: how data gets in (does it connect to your apps automatically, or do you upload files yourself?) and how the agent searches (one search tool across everything, or separate tools per app?).

We tested four approaches across these dimensions.

What we tested

We built a fake startup’s knowledge base. 91 Google Drive docs. 12 Slack channels with 836 messages. 441 GitHub files. Three months of realistic activity: product specs, incident threads, architecture decisions, code reviews, hiring discussions.

We wrote 99 questions that require information from at least two apps. The agent cannot answer from a single source.

Not every platform indexed the full corpus. ctxd indexed 100% of the content. The integration platform missed 43% of Slack threads and 7% of GitHub files. The RAG platform requires manual upload and does not connect to apps at all.

All four use the same AI model (GPT-5). Same files. Same questions.

How we measured quality

For each answer, an AI judge pulled out every factual claim and checked: does this fact appear in the data the agent actually retrieved? If a claim has no evidence in the search results, it does not count. We call these verified claims “grounded facts.”

Here is what the judge sees. A question goes in, the agent calls tools, gets data back, and writes an answer. The judge then reads the full trace and checks every claim.

Agent trace

QUESTION

"We missed the cert rotation for the Search API. What was the root cause and what fixes were proposed?"

↓

// Tool call: search

text:("cert rotation" OR "certificate" "search-api") application:slack

↓

RETRIEVED DATA

#incidents

Alice Chen: Residual risk low now that cert is rotated. Action items:

— Carol: file PR to move search-api mTLS to cert-manager, add renew-before: 30d

— Alice: add cert expiry checks to ops checklist. Alert at 30/14/7 days.

— Bob: draft customer comms.

This shouldn't have slipped during v2.0 focus.

↓

AGENT ANSWER

"The cert rotation for the Search API was missed because it slipped during v2.0 focus. The team proposed migrating to cert-manager with renew-before: 30d, adding expiry alerts at 30, 14, and 7 days, and implementing automated rotation across all services by end of Q2."

The judge reads the trace top to bottom. Every green highlight matches something in the retrieved data. The red highlight does not appear anywhere in the trace. The agent made it up. That claim is not counted.

An answer with 8 grounded facts contains 8 evidence-backed statements. An answer with 0 is the agent making things up.

The results

We measured two things. Left: can the agent answer questions that require information from multiple apps? (100 questions, grounded facts per answer.) Right: can the search engine find a specific document among 523 highly similar papers when the query uses different words? (100 queries.)

ctxd produced 4.8x the evidence-backed facts of the integration platform and 1.6x the official MCP servers. Zero errors across 356 tool calls. On pure search, ctxd found the right paper 94% of the time vs 52% (integration platform), 49% (RAG), and 47% (official MCPs via Google Drive).

What the agent actually gets back

The format of search results determines what the agent can reason over. ctxd returns full text with keyword highlights, source paths, and channel names. This is a deliberate design choice. The results are structured for LLM consumption, not human browsing.

Most alternatives return something the agent cannot use directly. Metadata with no text. Raw CSV with internal IDs. Relevance scores with opaque URIs. The agent either makes follow-up calls to read the actual content or gives up and hallucinates.

Here is the same question asked to ctxd and the integration platform:

“How is the backfill process designed to avoid impact on permissions or embedding quality during reindexing?”

Same question. ctxd returned 77KB of relevant Slack discussions from 6 channels in one call. The integration platform made 7 calls, got mostly empty results, and returned a generic documentation page about reindexing that had nothing to do with the team’s actual backfill process.

The official MCP servers have a different problem. The Slack MCP returns raw CSV with internal user IDs. Google Drive returns file metadata with no text at all.

What breaks agent retrieval

Three things cause agents to fail at retrieval. Tool confusion is the most visible. But lossy results and incomplete ingestion are just as damaging.

Tool confusion

The more tools you give an agent, the worse it performs. Not because the tools are bad. Because the agent gets confused.

When an agent sees 76 tools, it does not know which one to use. It picks the wrong tool. It invents parameters that do not exist. It wastes turns and context window on tools that return nothing.

What happens when the agent sees 76 tools

One platform exposes 76 tools by default. Delete buttons, ingestion tools, connectors for apps you do not use, and a built-in AI answer generator. Every tool definition is a chunk of JSON schema that goes into the model’s context window. 76 tools means thousands of tokens spent before the agent reads a single word of your question.

We ran the same benchmark three times on this platform, stripping tools each time. The agent broke in different ways at each level:

ctxd ships 3 tools. No filtering needed. 8.2 grounded facts per answer, 89.4% backed by evidence, zero errors.

What the agent did with 76 tools

With all 76 tools visible, the agent on one query:

Four tool calls. Zero useful results. The agent looked busy. It accomplished nothing.

Three query languages is two too many

Each official MCP server uses the query format of the app it wraps. Connect all three and the agent has to learn three different syntaxes:

Google Drive

fullText contains 'GDPR'

Slack

GDPR data deletion

GitHub

q: "GDPR in:title,body"

The agent mixes them up. It writes uppercase OR for Google Drive. The Drive API rejects it:

The agent retries, wasting a turn. These syntax mistakes account for most of the 10.5% error rate.

ctxd uses one query language across all apps:

Lossy results

Even when the agent calls the right tool, the response format can kill it. The Slack MCP returns raw CSV with internal user IDs like U0AJMTTET6W. The Google Drive MCP returns file metadata with no text. The agent has to make a second call just to read the content. Each extra call burns tokens and risks losing the thread.

The integration platform returns opaque internal URIs and relevance scores as low as 0.0167. No keyword highlights. No channel names. The agent cannot tell if a result is relevant without reading the full document.

ctxd returns full text with keyword highlights, source paths, and channel names. One call. The agent reads the result and reasons over it immediately.

Incomplete ingestion

You cannot find what was never indexed. The integration platform missed 43% of Slack threads and 7% of GitHub files. On questions where the answer lived in those missing threads, the agent had no chance regardless of how good its query was.

ctxd indexed 100% of the content across all three apps. Complete ingestion is not a feature. It is a prerequisite.

Same question, three different outcomes

Query: “How does the switch to text-embedding-3-large influence vector search performance and cost, and what safeguards were suggested during rollout?”

The answer lives across four Slack channels and a Drive RFC. No single source has it all.

Retrieval is a programming problem

Strip the integration platform down to 1 tool. Remove all tool confusion. The integration platform still produces just 1.7 grounded facts per answer; ctxd produces 8.2. That 4.8x gap has nothing to do with tool count — it is architectural.

Agents query. They do not search.

ctxd gives agents a domain-specific query language. Boolean operators, wildcards, phrase matching, app-level filters. Queries compile to an AST. The agent writes one precise query and gets exactly what it needs.

The integration platform’s agent made 29 progressively broader natural-language searches on one question and found nothing. ctxd’s agent wrote one structured query and got 14 verified facts.

Results are structured for code

ctxd returns full text with keyword highlights, channel paths, and source URLs. The agent parses structured data and reasons over it in the same turn.

Google Drive MCP returns file names with no text. Slack MCP returns raw CSV with internal user IDs. The integration platform returns opaque URIs with relevance scores as low as 0.0167. In each case the agent needs follow-up calls just to read the content. Or it gives up and hallucinates.

Complete sync. No manual upload.

ctxd connects via OAuth and ingests automatically. 100% of Slack threads. 100% of GitHub files. Live sync keeps the index current as files change.

The integration platform missed 43% of Slack threads and 7% of GitHub files. The RAG platform requires you to upload documents yourself. No connectors. No sync.

One syntax across every app

Three official MCP servers means three query languages. The agent mixes them up. 10.5% error rate.

ctxd uses one DSL across Drive, Slack, and GitHub. Zero errors across 356 calls.

3 tools. Any agent. Minutes to connect.

ctxd ships an MCP server with 3 tools: search, fetch_document, get_profile. Any MCP-compatible agent connects in minutes. No custom integration code. No SDK to learn. Works with Claude, ChatGPT, Cursor, or your own agents.

Takeaways

How we ran this

All approaches connect to the AI model via the Model Context Protocol (MCP). Each backend runs as a local process. The AI judge checks every factual claim against the data the agent actually retrieved. General knowledge without evidence from a tool call is rejected.

The test corpus was designed so answers require multiple apps. No question can be answered from a single document or channel.

Limitations

• Single AI model (GPT-5). Results may differ with other models.
• Synthetic test data. Your company data might behave differently. Contact us to find out.
• Single run per question. Results may vary across runs.
• We filtered each platform to its search tools only. In production, users face the full, unfiltered tool set.

Make your agent better at search

ctxd connects to your agent via MCP in minutes. 3 tools. No custom integration code. Works with Claude, ChatGPT, Cursor, or your own agents.

We run focused pilots on real enterprise workflows. Contact us to find out what your agents are missing.