5 min read
Same model, different agent: 17 points apart on 21 MCP servers
Claude Haiku scored 82% on 21 MCP servers through OpenCode and 65% through Claude Code. The gap comes from the built-in tools each agent reaches for first.
We ran the same model, Claude Haiku 5.5, through two coding agents against the same 21 MCP servers, with the agent as the only difference. Through OpenCode it answered 82.0% of the tasks correctly from the server. Through Claude Code it answered 65.4%.
We were testing M3, our open-source tool for testing MCP servers, and wanted a workload big enough to break it: 21 real servers, about 20 tasks each, three agents, three trials per task. The results became the MCP Agent Usability leaderboard, and every number in this post comes from it.
The gap lives on 10 of the 21 servers
Our first guess was that OpenCode is simply better at calling tools. Splitting the servers by what they do rules that out.
Eleven of the servers do a job none of the agents has a built-in tool for: a Postgres database, MongoDB, Redis, Kafka, Linear, Jenkins, Grafana, Kubernetes, Gitea, and two browser servers. The other ten overlap with something every agent already carries: web search and docs lookup (Exa, Fetch, Microsoft Learn, DeepWiki, Terraform), files and shell (Git, Filesystem, Desktop Commander, Serena), and memory.
| Servers | Claude Code (Haiku 5.5) | OpenCode (Haiku 5.5) | Codex (GPT-6 Luna) |
|---|---|---|---|
| 11 with no built-in overlap | 93.5% | 96.2% | 92.1% |
| 10 that overlap a built-in | 34.5% | 66.3% | 17.2% |
| All 21 | 65.4% | 82.0% | 56.5% |
On the first eleven, the three agents land within 4.1 points of each other, even though Codex runs a different model. On the other ten, Claude Code trails OpenCode by 32 points and Codex trails it by 49.
One Exa task, two right answers
Here is one task from the Exa server, run once through each agent. The prompt doesn't name a tool:
Every Kubernetes release has a release name and logo theme. What is the release name of Kubernetes v1.37, released in August 2026?
OpenCode called Exa's web_search_exa, found the release post on kubernetes.io and answered "Garhwal". Claude Code used ToolSearch to load its own WebSearch tool, searched, found the same post and also answered "Garhwal".
The leaderboard scores the first as a success and the second as a miss. It asks a narrower question than "was the answer right": did the agent get the answer from your server? An agent that solves your user's problem without touching your server is a fine result for the user, and it tells you nothing about whether your server works for them.
Most of Claude Code's 17-point deficit is right answers that came from somewhere other than the server. On the ten overlapping servers, Claude Code answered 378 trials through its own tools, and 342 of those answers contained the expected value. OpenCode answered 195 that way.
What each agent reaches for instead
Claude Code went to WebSearch and WebFetch for anything that looked like a web question, and to Bash and Read for files. On the Memory server it often skipped the server and opened MEMORY.md, the notes file of Claude Code's own memory feature, looking for the answer there.
Codex is more single-minded. On Microsoft Learn, Terraform, DeepWiki and Exa it didn't call the server in a single should-call trial. It searched the web with its built-in web_search instead, and sometimes used the shell. Averaged over all 21 servers, Codex skipped the server on 44.5% of the trials where it should have called it, against 36.5% for Claude Code and 16.1% for OpenCode.
OpenCode's advantage comes mostly from a tool it doesn't have. In 1,239 trials it never called a built-in web search tool, and the only web built-in it used was webfetch. Give it a question that needs a search and the MCP server is the only search it has, which is why it scored 96% on Exa while the other two scored 11%.
Files and shell are the exception, because every agent has a shell. All three mostly ignored Git, Filesystem, Desktop Commander and Serena, and OpenCode's lead shrinks to between 8 and 37 points there.
If you ship an MCP server
Your server competes with the tools the agent brings along, as well as with other servers. A docs or search server can work perfectly and still never get called by Codex.
That also means a single number for "how good is this server" depends heavily on which agent produced it. If your users mostly run Claude Code, test through Claude Code. If they run several agents, test through each one, because the same server, model and task can score 96% on one and 11% on another.
The leaderboard shows an ignore rate next to every success rate for this reason. A low score with a high ignore rate means the agent never asked; a low score with a low ignore rate means it asked and something went wrong.
How we measured
We ran Claude Code 2.1.312 and OpenCode 1.19.5 on claude-haiku-5-5, and Codex 0.165.0 on gpt-6-luna, between 29 October and 1 November 2026. Each of the 21 servers has about 20 tasks in seven shapes (lookups, searches, multi-step chains, writes, error recovery, collisions with built-in tools, and tasks where the server shouldn't be called), three trials per task per agent: 3,717 scored trials in all. Scoring follows spec 1.6.0.
A trial counts as a success when the answer is correct and its key value appears in a server response earlier in that trial. Local servers run on fixtures seeded with random values per trial, so a model can't answer from memory. Before a task is scored, it has to pass when the agent is told which tool to use, and fail when the server is missing. A quarter of each server's tasks are held out and never published.
The comparison has two limits. Codex runs a different model, so its column compares an agent and a model together; only the Claude Code and OpenCode columns isolate the agent. And "overlaps a built-in" is our grouping of the servers, based on what each server does rather than on the scores.
Every cell, interval and public transcript count is on the leaderboard; the Exa page shows the starkest split. To run the same tests on your own server through the agents your users have, start with the M3 documentation.