Pact

Inspiration

At the 8090 workshop, two things stuck with us: understanding the people who use a legacy system is the hardest part of modernization, and users are bad at giving feedback. Software Factory already checks code against blueprints and blueprints against requirements, but nothing checks requirements against the people they were written for. That's where modernization projects fail: a clerk forgets to mention a ten-year-old workaround, the new system quietly removes it, and nobody notices until work piles up. We wanted a tool an 8090 engineer would want on day one of an engagement.

What it does

Pact turns the people who use a legacy system into requirements that can be checked.

During a client interview, an 8090 engineer wears Meta glasses while the laptop records. An AI agent that has read the client's legacy COBOL, docs and usage logs suggests the next question, including questions about business rules in the code that nobody has mentioned. A tap on the glasses plays it in the interviewer's ear.

Every answer feeds a behaviour contract: a checkable statement of how a requirement should show up in real usage. Each is backed by the interviewee's own words, the question that prompted them, and a confidence level based on whether what people said, what the logs show and what the code enforces agree. Pact drafts contracts from the interview and an 8090 reviewer approves them.

After launch, and only if the client opts in, Pact checks real usage against the contracts. It reports by role, never by person, and patient data never leaves the browser. When people drift from a requirement, Pact shows which promise broke, what it costs, and which interview moment it came from. In our demo, billing clerks start typing prior-authorization numbers back into the notes field because the new field is hidden on a second tab.

Each contract lands in Software Factory as a "Behaviour contract" section of the Feature Requirements doc (Markdown export), and as a Feedback item with a structured header when it breaks. We also wrote skills so Software Factory's own agents can write, triage and review contracts.

Our demo client is a fictional Medicare billing office. All data is synthetic and the Software Factory connection is simulated.

How we built it

Interviewer client: a Chrome page that transcribes with Chrome speech recognition, reads the glasses' taps as media-key events (single: next question, double: skip, triple: repeat), and plays questions through the glasses.
Interview engine: FastAPI with Claude Haiku 5.5. After each interviewee sentence, two parallel calls run on a cached brief: one picks the next questions (mostly from 50 pre-voiced ones), the other records which contract slots were filled and any contradictions.
Legacy company in a box: a small COBOL claims program with seven planted business rules, an outdated 2009 manual, role-play scripts for three staff members, and a hidden answer key to score our agent.
Contracts: a JSON Schema with five shapes (field usage, time, path, sequence, absence), drafted by Claude and approved by a human.
Drift detection: a seeded simulator generates 12 weeks of legacy usage and 8 weeks after launch. The detector checks each contract weekly, with a learning period, a two-weeks-in-a-row rule, a privacy floor of 5 people, and cost estimates.
Evidence board: plain HTML, CSS and JavaScript showing the live interview, contracts and evidence, drift charts, and the Software Factory hand-off.
How we worked: four teammates, each running their own coding agent (Claude Code or Codex) from a per-role CLAUDE.md / AGENTS.md, with git tags as gates and one integrator merging everything.

Challenges we ran into

The glasses: Meta's toolkit doesn't expose tap events. The glasses send standard media keys, but only when used as headphones; using their microphone switches them to call mode and the taps stop. So the laptop records and the glasses only speak and listen for taps.
Latency: an interviewer can't wait for a model. We split each turn into a fast question call and a slower bookkeeping call, cached the brief, and pre-voiced the prepared questions.
Proving we weren't fooling ourselves: since we planted the drift, we added a holiday-week spike decoy the detector must ignore, and graded our agent against a hidden answer key.
Integrating without access: with no Software Factory account, we built against their public docs, including the newer Feedback Integration API.
Coordinating four agents: six tasks all wanted the same server file, so we split routes into one file per owner and made gates explicit.

Accomplishments that we're proud of

A live interview where a code-aware question arrives through the glasses in [fill: measured sentence-to-question time; during the build we measured 1.2–1.8 s].
Contracts that trace every field back to a quote and the AI question that produced it.
Our agent found [fill: X of 7] planted business rules, including [fill: Y of 4] hidden ones, scored against an answer key it never saw.
The drift detector caught the planted prior-auth drift from week 4 and ignored the holiday decoy.
AI-drafted contracts that pass schema validation and are checked against the transcript before a human approves them.
[fill: N] passing tests, built by four people in parallel without merge chaos.

What we learned

Detecting drift is the easy part. Tying it to a specific requirement and the person who described it is what makes it useful.
The best questions come from what's missing in a conversation. Legacy code is a better checklist than any template.
Verification is what lets an agent work with less supervision.
Being honest about scope (synthetic data, a simulated connection, a decoy in the data) made our claims stronger.

What's next for Pact

Read-back confirmation: interviewees confirm each contract in plain English before the meeting ends.
Pre-launch checks: run our persona-review skill (an agent acting as the real billing clerk) on every pull request through Software Factory's QA agent.
Native Software Factory integration: register as a fourth drift automation, comment-only and human-approved, posting Feedback items and suggestions through their API.
Real usage data: read events from tools like PostHog instead of our own format.
Video-call interviews: the same engine on Google Meet or Zoom audio.
Generated skills: turn each client's contracts into a QA skill for coding agents.

Built with

Python, FastAPI, Claude Haiku, Anthropic API, JavaScript, HTML, CSS, Chart.js, Web Speech API, Media Session API, ElevenLabs, Meta Ray-Ban glasses, COBOL, JSON Schema, pytest, GitHub, Claude Code, Codex