Crucible

• 

  git status, git log --oneline --decorate -8, npm test → Project state, commit history, and full test-suite verification

• 

  npm test → Conformance and chaos testing verified

• 

  npm test → 40 automated tests passing

• 

  npm run build / node packages/cli/dist/index.js --help → CLI build and command-line interface verification

• 

  node packages/cli/dist/index.js scan --help → Crucible scan command and CLI usage

Inspiration

As AI agents increasingly rely on tools and external data through the Model Context Protocol (MCP), we noticed an important gap: making an MCP server work is not the same as making it reliable.

A server might pass a basic request successfully but crash on malformed input, hang on an unsupported method, or behave incorrectly under protocol edge cases.

We wanted to build a tool that answers a harder question:

"What happens to an MCP server when things go wrong?"

That idea became Crucible.

What it does

Crucible is an open-source conformance and resilience testing toolkit for MCP servers.

It combines two complementary approaches:

• Conformance testing — verifies that an MCP server follows expected protocol behavior.
• Chaos testing — deliberately triggers failure scenarios and measures how the server responds.

Crucible classifies failures into meaningful outcomes such as:

• resilient
• degraded
• hung
• crashed

It also detects different MCP protocol eras and can test servers through supported transports such as stdio and Streamable HTTP.

The goal is not simply to determine whether a server works, but to determine how reliably it behaves when something unexpected happens.

How we built it

Crucible is built entirely in the TypeScript and Node.js ecosystem.

The project is structured as a monorepo with separate packages for:

• the low-level MCP client and process harness
• protocol conformance checks
• chaos and resilience scenarios
• CLI tooling
• reference fixture servers
• integration tests

The low-level client handles MCP communication, process lifecycle management, request timeouts, transport behavior, and follow-up liveness probes.

On top of that, the conformance engine executes protocol checks while the chaos engine runs controlled failure scenarios.

We also built intentionally broken fixture servers so that Crucible can test its own ability to detect crashes, hangs, degraded behavior, and successful recovery.

The project is continuously validated through automated builds and tests.

Challenges we ran into

One of the hardest problems was testing failure behavior without allowing the test runner itself to become unreliable.

A crashed process, a deliberately frozen process, and a slow but still responsive process can look very similar from the outside.

We therefore had to carefully design:

• process lifecycle handling
• timeouts
• cleanup and termination
• follow-up liveness probes
• failure classification
• transport-specific behavior
• protocol-era detection

Another challenge was avoiding false positives. A test should fail when a server violates the protocol, but it should not report a healthy server as broken simply because a particular check is not applicable to its transport or protocol era.

Accomplishments that we're proud of

We are proud that Crucible can go beyond ordinary request/response testing and actually evaluate server behavior under failure.

The current test suite validates scenarios including:

• malformed-request crashes
• unknown-method hangs
• frozen processes
• degraded responses
• successful resilience
• legacy and modern MCP protocol behavior
• HTTP and stdio behavior
• protocol conformance failures

We also built the testing infrastructure itself with intentionally broken fixtures, allowing Crucible to prove that it can correctly distinguish different failure states.

Most importantly, the project is open source and designed as reusable developer infrastructure rather than a one-off demonstration.

What we learned

Building Crucible taught us that reliability testing is fundamentally different from functional testing.

A successful response only tells us that one path worked. It does not tell us what happens when the input is malformed, the protocol changes, a method is unsupported, or the server stops responding.

We learned a lot about MCP, JSON-RPC, TypeScript, Node.js process management, automated testing, protocol compatibility, and chaos engineering.

We also learned that the infrastructure used to test software needs to be extremely defensive itself. Otherwise, it becomes difficult to tell whether a failure came from the system being tested or from the test harness.

What's next for Crucible

The next step is to expand Crucible from a testing toolkit into a broader reliability platform for MCP-based AI systems.

Planned directions include:

• more MCP conformance checks
• additional chaos scenarios
• richer machine-readable reports
• CI/CD integration
• improved developer diagnostics
• historical reliability tracking
• automated failure analysis
• AI-assisted explanations and remediation suggestions

Our long-term goal is to make Crucible a standard reliability layer that developers can run before deploying MCP servers into real AI-agent environments.

Built With

• agents
• ai
• automated
• chaos
• cli
• conformance
• context
• developer
• engineering
• http
• infrastructure
• json-rpc
• mcp
• model
• node.js
• open
• protocol
• resilience
• source
• stdio
• streamable
• testing
• tools
• typescript