Context MRI

Why I built it

AI agents do not only fail because they lack context. They also fail because their context contains an obsolete README, duplicated rules, irrelevant examples, or tool descriptions that contradict one another. Developers usually debug this by rereading prompts and guessing.

I built Context MRI to turn that guesswork into a controlled experiment.

What it does

Context MRI takes an agent task and its context bundle, establishes a full-context baseline, then removes one context item at a time. Every condition is repeated and scored against the same task-specific rubric.

The result is an ablation matrix showing which files help, do nothing, or actively make the agent worse. Every score opens into an inspectable trace with its run ID, prompt hash, rubric breakdown, latency, token usage, model output, and provenance.

In the included scenario, an archived guide tells the agent to use /v1/chat/completions while the current tool schema says /v1/responses. The baseline averages 43/100. Removing the stale guide averages 92/100 and improves all three paired repeats. Context MRI then builds a smaller recommended context pack and independently verifies it at 92/100 with 44% fewer context tokens.

How I built it

The interface is React and TypeScript. The experiment engine has a local Node/Express path and a public Cloudflare-compatible fixture adapter. The live path uses the OpenAI Responses API with GPT-5.6 Sol, medium reasoning, and strict Structured Outputs.

A fixed application evaluator scores endpoint accuracy, recency reasoning, legacy-risk handling, conflict explanation, and schema validity. The default experiment produces 18 discovery traces plus three separate pack-verification traces. Headline metrics and file classifications are calculated from those records rather than hard-coded into the interface.

The public judge demo uses a clearly labeled deterministic fixture replay so it works without an account, API key, or external setup. The repository includes the fresh GPT-5.6 path for funded OpenAI API projects.

How Codex helped

I used Codex throughout the project: researching the official requirements, challenging and narrowing the product idea, designing the experiment architecture, implementing the API and interface, writing tests, checking the math, improving the visual system, deploying the public demo, and running browser-based production QA.

The most valuable part was not raw code generation. Codex helped me notice where the product was overstating its evidence. I removed an early confidence claim that three repeats could not support, corrected inconsistent metrics, and made live-versus-fixture provenance visible in the product.

What I learned

Context quality is not the same as context quantity. The most damaging file can look authoritative. Simply cutting tokens is not enough; you need representative evals to show what can safely be removed.

What is next

The next steps are multi-item interaction ablations, imported production trace sets, human-calibrated rubrics, CI regression gates, and adapters for common agent frameworks. The longer-term idea is a context compiler that routes only the minimum proven context to each task.

Built With

• cloudflare-workers
• codex
• express.js
• gpt-5.6-sol
• openai-responses-api
• react
• typescript
• vite