Inspiration
Falsify started from my work on Kaggle hackathons and machine learning competitions. I wanted a system that could analyze a target model, explore several research branches in one session, and keep each branch grounded in its own evidence. The practical goal was to reduce context contamination and hallucination as a long research session accumulated hypotheses, failed attempts, and partial results.
A frontier GPT model using Ultra reasoning and subagents can already explore several branches in parallel. That helped with throughput, but it did not solve visibility. When multiple paths were active, I still could not answer simple questions at a glance: Which configuration produced the best result? Which branch failed for a technical reason? Which idea had already been tested? What should run next?
Conversation logs were too linear for this job. I added an experiment graph so that model lineage and outcomes would be visible, then introduced tickets as an active reservation queue for research ideas. A person can record a hypothesis before stepping away, while a main orchestrator or a Codex agent connected through WebMCP can read the current progress and understand what the ticket is asking for.
The longer-term goal is an overnight research loop in which the agent uses the accumulated evidence to refine or rewrite the next ticket when its assumptions are no longer valid, then continues with the correct experiment instead of blindly following stale instructions. The current browser MVP establishes the state, provenance, and approval rules needed for that workflow.
What changed for this submission
The earlier project focused on coordinating multi-branch research inside a single session. For the WebMCP Challenge, I built a browser-based research control plane that turns that progress into shared, inspectable state.
I added the ticket queue, experiment lineage graph, frozen research contract, deterministic browser runner, evidence comparison views, human approval center, and 12 task-specific WebMCP tools. Human actions and agent actions now operate on the same workspace instead of producing separate conversation histories.
What it does
Falsify is a browser-based research workspace operated by a human and a WebMCP agent through the same shared state.
Every project begins with a research contract that defines:
- the objective and primary metric
- the evaluation protocol
- the experiment budget
- the minimum effect required to count as an improvement
- the configuration keys an experiment may change
Once the baseline produces valid evidence, the contract freezes. Ideas enter the workspace as tickets. A claimed ticket becomes exactly one experiment node under a frozen parent. Running the node produces evidence, and a decision can then promote it, discard it, or create follow-up tickets.
This structure prevents a result from being reinterpreted after the fact. It also preserves the path from an idea to its configuration, run, evidence, and final decision.
Technical failure and scientific failure are handled differently. If a run crashes or diverges before producing a valid measurement, the node becomes repairable. It is not labeled as a regression because the hypothesis was never properly tested. A valid but worse measurement, on the other hand, remains part of the scientific record.
How I built it
This submission is a browser-only MVP. It has no backend, user account, remote runner, or arbitrary code execution.
The interface is built with React, TypeScript, Vite, Tailwind CSS, and shadcn/ui. Project state is stored in IndexedDB. Experiment training runs in a Web Worker so that the interface remains responsive.
The demonstration uses a deterministic two-moons classification problem and a small multilayer perceptron implemented from scratch. A fixed seed and frozen evaluation contract make the results repeatable during judging.
The application registers 12 focused WebMCP tools on document.modelContext. They cover the complete research loop:
- inspect the project
- list and create tickets
- claim a ticket
- materialize an experiment
- start or cancel a run
- inspect evidence
- compare nodes
- commit a decision
The tools call the same application service used by the React interface. Agent actions therefore appear in the human workspace without maintaining a separate copy of the project.
Falsify deliberately exposes no generic shell, file, or delete tool. Mutations are checked against the frozen research contract, project revision, and idempotency key before execution.
In Copilot mode, starting a run and committing a decision return an approval request instead of executing immediately. The proposal appears in the Approval Center, where a person can inspect and approve it. Manual mode applies approval gating to every agent mutation.
Challenges
The first challenge was preserving one consistent state machine across the human interface and WebMCP tools. A command had to obey the same rules regardless of whether it came from a button or an agent call.
The second challenge was handling retries and concurrent actions safely. Each mutation includes an idempotency key and expected project revision. Retrying the same request replays its original result, while stale commands are rejected instead of silently overwriting newer work.
The third challenge was separating technical failure from scientific evidence. A diverged run must remain repairable, while a valid negative result must remain immutable. Treating both outcomes as a simple failed status would undermine the research record.
The final challenge was making the demonstration deterministic without hiding the real workflow. The benchmark is intentionally small, but it still performs actual training in a Web Worker and produces metrics, curves, configuration differences, logs, and a decision boundary.
What I learned
I learned that an agent tool catalog is more than an API surface. Its names, descriptions, schemas, approval behavior, and output annotations collectively define what the agent believes it is allowed to do.
I also learned that sharing storage is the easy part of human-agent collaboration. The harder part is sharing meaning: which contract governed a run, why a node exists, whether a measurement is valid, and who approved the next step.
Explicit state transitions, frozen inputs, provenance links, and narrow tools made those boundaries easier for both the human and the agent to understand.
What is next
The next version will connect the same research model to local repositories and real experiment runners. Planned work includes a local daemon, SQLite and artifact storage, one Git worktree per experiment node, coding-agent adapters, and remote execution.
The current browser profile remains intentionally narrow. It demonstrates the complete measure, compare, and decide loop without granting an agent generic access to the user’s machine.
Built With
- indexeddb
- playwright
- pnpm
- react
- shadcn/ui
- tailwind
- typescript
- vite
- vitest
- web
- webmcp
- zod
Log in or sign up for Devpost to join the conversation.