# How to test an MCP server with Codex

> Codex is the agent most likely to skip your server: in our runs it ignored it on 44.5% of the trials where it should have called it. SineFrame M3 runs the Codex CLI against your server and checks the tool calls it records, so you find out before your users do.

Facts last checked: 2026-11-17.

## Short answer

Run Codex against your server inside a pytest test, restrict it to the tools you name, and assert on the recorded call. With SineFrame M3:

1. Install the M3 CLI with uv tool install sf-m3-cli, then run m3 init and m3 setup.
2. Select the codex harness with OPENAI\_API\_KEY or CODEX\_API\_KEY, or reuse your ChatGPT login.
3. Call agent.run(...) with tools=\["server:tool"\] and assert with expect(result).to\_have\_tool\_call(...).
4. Pin the Codex version with m3 test --runtime managed --harness codex@VERSION=MODEL.
5. Check the schema with m3 lint --harness codex, because Codex rewrites large schemas.

## What does a Codex test look like?

The test starts your server, gives Codex one goal, limits it to the shipping\_quote tool, and asserts on the call M3 recorded: tool name, server, arguments, a successful result, and exactly one call.

From the harness guide:

```
import os
import sys
from pathlib import Path

from m3 import ExecutionOutcome, MCPTestKit, expect
from m3.types import StdioServer

HERE = Path(__file__).resolve().parent


def test_codex_calls_the_shipping_tool() -> None:
    server = StdioServer(
        name="shipping",
        command=sys.executable,
        args=(str(HERE / "shipping_server.py"),),
        cwd=str(HERE),
    )
    with MCPTestKit(env={}) as kit:
        agent = kit.agents(
            [
                {
                    "harness": "codex",
                    "models": [os.environ["M3_DOCS_CODEX_MODEL"]],
                    "credential_env": {"OPENAI_API_KEY": "MY_OPENAI_KEY"},
                }
            ]
        )[0]
        result = agent.run(
            "Call shipping_quote once for weight_kg 2 in zone local.",
            server=server,
            tools=["shipping:shipping_quote"],
            timeout=120,
        )

    assert result.snapshot.outcome is ExecutionOutcome.COMPLETED, result.error
    expect(result).to_have_tool_call(
        "shipping_quote",
        server="shipping",
        arguments={"weight_kg": 2, "zone": "local"},
        status="success",
        count=1,
    )
```

## How does Codex log in during a test?

Map OPENAI\_API\_KEY or CODEX\_API\_KEY and M3 passes the key to Codex’s login flow, kept in memory only. With no mapping, M3 copies the auth.json from your Codex login, and credential\_env={} forces that reuse of a ChatGPT login. When neither exists, startup fails with "Codex has no login".

## How are Codex approvals handled?

M3 answers Codex’s MCP tool approval prompts from the test’s tools selection: a tool outside the selection is declined, even when permission\_policy is "allow". Command, sandbox, and file approvals go to permission\_policy, which denies by default, and the trace records each one as a request and response pair. Set Codex’s sandbox and permission profile with Codex(permissions=...).

Agent-driven elicitation is supported on Codex and tested on version 0.156.1, with at most 9 rounds per action.

## What does Codex do differently with MCP servers?

Codex prefers its own web search. Across 21 MCP servers, Codex 0.165.0 on GPT-6 Luna scored 56.5%, and 17.2% on the 10 servers that overlap a built-in tool. On Microsoft Learn, Terraform, DeepWiki, and Exa it did not call the server in a single trial that needed it.

Codex also changes what the model sees. It compacts input schemas over 5,000 bytes and removes descriptions first; in our tests a 14 KB schema lost its required nested fields, and the call came back once the schema budget was raised. Codex does not act on list\_changed, so a tool added after startup was never called. And one prompt line, "If a tool is not available, say which one and stop", broke every Codex run.

- 21 servers: Codex 56.5% (Claude Code 65.4%, OpenCode 82.0% on Haiku 5.5).
- Should-call trials where Codex skipped the server: 44.5%.
- Well-built vs flawed issue-tracker server: GPT-6 Luna 85.6% vs 50.0%, GPT-6.1 Sol 99.0% vs 95.0%.

## How do I pin a Codex version?

Pass --runtime managed with a selector of the form codex@VERSION=MODEL. M3 downloads that release from the openai/codex repository and records the requested and resolved version with each execution. Name two Codex selectors to compare versions of the same agent on the same tests.

## Frequently asked questions

### Why does Codex not call my MCP server?

Usually because a built-in tool can do the job: Codex used its own web search instead of search and docs servers in our runs. It can also miss tools whose schemas it compacted or that appeared after startup. An M3 agent test makes each case a failing assertion.

### Can I use my ChatGPT login instead of an API key?

Yes. Set credential\_env={} and M3 reuses the auth.json from your Codex login. With OPENAI\_API\_KEY or CODEX\_API\_KEY mapped, it uses the key instead and keeps it in memory.

### Will Codex run shell commands during my test?

Only if you allow it. Command, sandbox, and file approvals go to the test’s permission\_policy, which denies by default, and each request and answer is recorded in the trace.

## Related

- [How to test an MCP server](https://m3.sineframe.com/mcp-testing)
- [Test an MCP server with Claude Code](https://m3.sineframe.com/mcp-testing/claude-code)
- [Test an MCP server with OpenCode](https://m3.sineframe.com/mcp-testing/opencode)
- [Write MCP tool descriptions that agents call](https://m3.sineframe.com/mcp-tool-descriptions)
- [Research: Claude Code vs Codex on 10 MCP features](https://m3.sineframe.com/blog/claude-code-vs-codex-mcp)
- [Choose an agent harness](https://m3.sineframe.com/docs/guides/agents/harnesses)
- [Compare harness versions](https://m3.sineframe.com/docs/guides/agents/versions)
