MCPJam
The testing & evaluations platform for MCP servers
MCPJam tests, debugs, and evaluates MCP servers the way real AI clients use them: inspector and playground, a client matrix emulating ChatGPT, Claude, Cursor, and Copilot, user testing via shareable chatboxes, agent-swarm acceptance tests, evals, and CI/CD gates. The core client, Inspector, CLI, and SDK are open source and free; paid plans add the live client matrix, hosted swarm/UAT, AI insights, and team reporting. The site claims 50,000+ developers and 200+ enterprises (case studies: Asana, Bright Data); the PH launch page claims 106,000+ developers.
What is MCPJam?
MCPJam is a testing, debugging, and evaluation platform for MCP (Model Context Protocol) servers. It exercises an MCP server the way external AI clients do — an inspector and playground for request-by-request reproduction, a client matrix of maintained emulations of ChatGPT, Claude, Cursor, Copilot and more, swarm persona testing, shareable UAT chatboxes, evals, and CI/CD gates — so teams can verify that users actually get their intended outcome across clients before release. The core client, Inspector, CLI, SDK, local evals, and conformance checks are open source and free; paid plans add the live client matrix, hosted swarm/UAT, AI insights, reporting, and enterprise governance.
Key features
- Inspector & Playground: reproduce MCP failures request by request; run local servers via desktop app, CLI, or SDK (npx @mcpjam/inspector)
- Client matrix of continuously maintained AI-client emulations (ChatGPT, Claude, Cursor, Copilot) so host changes don't quietly break tests
- Evals in CI/CD: gate merges on outcome-based evals (right tool chosen, right arguments, job completed) rather than assertions
- Swarm: AI agent personas that run multi-turn acceptance journeys across clients to find where workflows break
- Chatboxes: hosted, shareable UAT environments that capture and replay human tester sessions as regression tests
- Token-level traces per step across clients plus a tokenizer framework for per-workflow cost estimates (per maker responses on launch page)
- OAuth, XAA/EMA, scoped-permission and step-up-authorization debugging
- SDK- and framework-agnostic; strictly pre-production — it does not instrument your code or live traffic
Who it's for
- Teams shipping an MCP server that must behave consistently across ChatGPT, Claude, Cursor, and Copilot
- Enterprises building agent-facing integrations (case studies named on the site: Asana, Bright Data, IBM, Apollo, Scalekit, Jeppesen ForeFlight)
- CI/CD regression gates for MCP tool changes before release
- Capturing and replaying real user sessions on agent surfaces for QA
- Quantifying per-workflow token cost across AI clients for procurement or budgeting
When not to use it
MCPJam is strictly pre-production: it does not observe production traffic or live user traces, and it evaluates how your server behaves inside external agents — not the agent you built inside your own app (agent-observability tools like Braintrust, LangSmith, or Datadog cover that side). It is not a model gateway or router.
FAQ
Is MCPJam open source or free?
The core is open source and free: the client, Inspector, core CLI and SDK, local evals, and conformance checks (github.com/MCPJam/inspector). Paid plans add the hosted live client matrix, swarm, chatboxes, AI insights, reporting, and enterprise governance (SSO, audit logs, DPA; SOC 2 Type 2 in progress).
What is the difference between MCP evals and agent evals?
Agent-eval tools measure the agent you built inside your own system. MCP evals measure the other side of the handshake: how your MCP server behaves when an external agent you don't control (ChatGPT, Claude, Copilot, Cursor) decides whether to call it, with what arguments, and how it uses the result.
Does MCPJam touch production traffic?
No. It is strictly pre-production: during dev, QA, beta, and CI/CD it exercises your server the way real AI clients do, without instrumenting your code or live traffic.
How does MCPJam test a non-deterministic system?
With evaluation, not assertions: it scores whether the agent selected the right tool, sent the right arguments, and completed the job across runs and clients, then diagnoses why a score dropped.
How do I get started?
Run npx @mcpjam/inspector@latest for the open-source inspector, or use the web app at app.mcpjam.com (the maker states single-client sanity checks run without sign-in).
Share this launch
Embed this badge
<a href="https://orangebot.ai/product/mcpjam" target="_blank" rel="noopener noreferrer"> <img src="https://orangebot.ai/api/badge/mcpjam.svg" alt="Featured on OrangeBot" width="200" height="54" /> </a>