SineFrameM3CI gate for MCPs Back to home
MCP testing with Claude Code

How to test an MCP server with Claude Code

A server that passes direct protocol tests can still go unused once Claude Code is the client. SineFrame M3 runs the Claude Code binary itself against your server, records every tool call it makes, and lets a pytest test assert on them.

Facts last checked November 17, 2026

Short answer

Run Claude Code against your server inside a pytest test and assert on the tool calls it recorded, not on its reply. With SineFrame M3:

  1. Install the M3 CLI with uv tool install sf-m3-cli, then run m3 init and m3 setup.
  2. Select the claude_code harness and give it ANTHROPIC_API_KEY, or map CLAUDE_CODE_OAUTH_TOKEN to use a subscription login.
  3. Assert on the recorded call with expect(result).to_have_tool_call(...).
  4. Pin the Claude Code version with m3 test --runtime managed --harness claude_code@VERSION=MODEL.
  5. Check the server first with m3 lint --harness claude_code.

How does M3 run Claude Code?

M3 drives Claude Code through a native adapter named claude_code. With the default system runtime it uses the Claude Code executable already installed on the machine. With the managed runtime, M3 downloads the exact version you name from Anthropic’s release manifest and verifies it against the checksum in that manifest, so every run records which binary produced it.

Each run gets an isolated home directory. Login files from your own Claude Code installation are not copied into it, so a test sees only the credentials you hand it.

The test limits the agent to the tools you name and records what Claude Code actually called. An assertion on that record fails when the agent never made the call, even if its reply claims it did.

Which credentials does Claude Code need in a test?

When a test sets no credential mapping, M3 forwards ANTHROPIC_API_KEY to Claude Code. To run on a Claude subscription instead, map CLAUDE_CODE_OAUTH_TOKEN, for example credential_env={"CLAUDE_CODE_OAUTH_TOKEN": "MY_CLAUDE_TOKEN"} in the SDK. A mapping names environment variables, never secret values, and an explicit mapping is the complete set of credentials the agent receives.

Agent tests call a model, so each trial costs what the model costs. Direct tests of the same server need no credentials at all, which is why they make the better pull-request gate.

How do I pin a Claude Code version?

Claude Code changes often, and its behavior with MCP servers changes with it. Pass --runtime managed with a harness selector of the form claude_code@VERSION=MODEL. latest is not a pin, and a version with a leading v is rejected. Name several selectors to compare versions or agents in one run.

From our Claude Code vs Codex post: two pinned agents, three trials each
m3 test --runtime=managed \
  --harness [email protected]=claude-sonnet-5-5 \
  --harness [email protected]=gpt-5.6-sol \
  --trials 3 -- quirks/

What does Claude Code do differently with MCP servers?

Claude Code reaches for its own tools first. With the same model, Claude Haiku 5.5, it scored 65.4% across 21 MCP servers where OpenCode scored 82.0%. On the 10 servers whose job overlaps a built-in tool, such as web search, docs lookup, files, and memory, it scored 34.5%, because it answered from WebSearch, WebFetch, Bash, or Read instead of the server. On the other 11 it scored 93.5%.

In our feature tests on Claude Code 2.1.287, three of ten MCP features failed in every trial: an argument named ids[], a marker in the middle of a 12,000-character error message (Claude Code keeps roughly the first and last 5,000 characters of an error), and a result that carried both text and structuredContent.

When a result includes structuredContent, Claude Code gives the model that and drops the text, so everything the model needs belongs there. Claude Code also applies one MCP startup timeout to every server, so servers with different startup budgets can fail together.

  • 21 servers, same model: Claude Code 65.4%, OpenCode 82.0%.
  • Servers that overlap a built-in tool: Claude Code 34.5%.
  • Should-call trials where Claude Code skipped the server: 36.5%.

How do I check a server before running Claude Code?

m3 lint reads your server’s tool metadata without calling a tool and reports what the targeted Claude Code version will do with it: names that are too long, descriptions it truncates, results it drops. With --directory claude it also applies Anthropic’s connector directory rules. It exits non-zero on a finding at or above --fail-on, so it works as a CI step.

From the directory readiness reference
m3 lint --directory claude --harness claude_code -- python server.py

Frequently asked questions

Can I test with my Claude subscription instead of an API key?
Yes. Map CLAUDE_CODE_OAUTH_TOKEN in the test’s credential_env or with --credential-env. Without a mapping, M3 forwards ANTHROPIC_API_KEY. Login files from your own Claude Code installation are not copied into the test.
Why did Claude Code answer correctly without calling my server?
It often uses its built-in tools when they can answer: web search for web questions, Bash and Read for files. An M3 agent test checks the recorded tool calls, so it fails in that case even when the answer is right.
Does M3 run Claude Code or simulate it?
It runs the real Claude Code executable, either the one installed on the machine or an exact version the managed runtime downloads and verifies.

Related

  • How to test an MCP server
  • Test an MCP server with Codex
  • Test an MCP server with OpenCode
  • Research: same model, different agent on 21 MCP servers
  • Research: Claude Code vs Codex on 10 MCP features
  • Choose an agent harness
  • Pin harness versions

Try it on your own MCP server

SineFrame M3 is free and open source (Apache-2.0). Direct tests need no model or API key, and runs stay on your machine unless you publish them.

Write your first MCP testInstall M3M3 on GitHub

© 2026 SineFrame M3
HomeMCP testingMCP CI gateMCP tool descriptionsM3 vs MCPJamM3 vs MCP InspectorBlogDocumentationPrivacy policyTerms of use