EditLoop

• 

  editloop

• 

  same-fix-again

• 

  draft-misses-signoff

• 

  edit-becomes-rules

• 

  read-or-delete-a-rule

• 

  held-out-43-to-14

• 

  one-miss-left

• 

  memory-you-can-see

Inspiration

If you use AI drafts for support replies, you end up fixing the same mistakes every day. The draft forgets the sign-off, promises a refund the policy doesn't allow, or doesn't send a billing dispute to the right person. You fix it, and on the next ticket you fix it again. Some tools do learn from your edits, but you can't see what they learned and you can't undo it. We wanted the memory to be something an agent can actually read.

What it does

An agent opens a ticket, gets an AI draft and edits it like they normally would. EditLoop compares the draft with the edit and writes up to three rules in plain English, for example:

Include a professional sign-off with the company name at the end of every response.

The rules show up in a panel. Later drafts use them, and the panel marks which rules each draft used and how often the agent kept the result. If a rule is wrong, you delete it.

To check that the rules carry over to tickets the system hasn't seen, we kept 4 tickets out of training. One button replays them with and without the learned rules. In our recorded run:

The demo uses a fictional subscription app called Brightbox, with 10 training tickets and 4 held-out ones.

How we built it

It's a Next.js app in TypeScript. Gemini (gemini-3.1-flash-lite) writes the drafts and pulls rules out of each edit, with OpenRouter as a fallback. The house policies are never shown to the model. Fixed pattern checks in code score every reply against them, so the model never grades its own work.

We wanted the public demo to work without anyone needing an API key. So we recorded one full run, and the deployed app replays it. Live model calls stay off unless you turn them on locally. The app is on Vercel and has 92 unit tests.

Challenges we ran into

Our first policy checks were wrong. They counted replies like "I can't guarantee a fix time" as broken promises. Fixing them changed which training drafts counted as misses, and that changed what the system learned, so we recorded the whole run again. The first run had shown 0% misses with rules. We threw that number out because it came from bad checks.

Free-tier rate limits slowed recording down, so the recorder waits when it gets limited and picks up where it stopped.

Accomplishments that we're proud of

The loop works end to end on tickets it never trained on, and we can show exactly why. Every rule it learned is a sentence on screen that you can check and delete.

What we learned

Rules pile up fast. Our run produced 12, and some are near-duplicates. Two of them contradict each other: one says to sign off as yourself, the other as the team. One held-out reply still missed the sign-off. We could only spot that because the memory is visible, but it also showed us that a rule list needs upkeep.

What's next for EditLoop

We want to merge duplicate rules and flag conflicting ones instead of keeping both. We also need to test on far more tickets and more than one run. Right now 4 held-out tickets and 7 checks show that the loop works, but not how well. After that, we'd connect it to a real helpdesk like Zendesk.

Built With

• gemini
• next.js
• openrouter
• react
• typescript
• vitest