GlitchGym

The debugging coach that breaks your code on purpose. AI injects one realistic bug, a sandbox proves it with failing tests, and a Socratic coach guides you to the fix without giving it away.

Comment

Inspiration

Every CS student now has an AI that fixes their bugs. Paste the traceback, copy the fix, move on. The error disappears, and so does the one moment where you would have learned something. Debugging is a skill you only build by debugging, and nobody gives students practice reps. Glitchathon's tagline is "Break things, fix them with AI", so I built something that does exactly that.

What it does

GlitchGym is a debugging coach with three parts:

  • 🩺 Diagnose (the Glitch Detective brief). Paste broken Python. GlitchGym runs it to capture the error, then explains why it broke: what the error means in plain English, the root cause at the exact line, a step-by-step trace of what Python did, a minimal fix, a rule of thumb, and a one-question quick check. The fix is executed in a sandbox before you see it, so you're told whether it actually works. One click turns the diagnosis into a practice drill on the same bug type.
  • 🏋️ Gym (the twist). GlitchGym takes code that works, and AI injects one realistic bug of a chosen type (off-by-one, mutable default, aliasing, and 7 more). The bug is only accepted after the sandbox proves the original passes every test and the broken version fails at least one. You fix it in the browser, with a 3-step hint ladder and a Socratic coach that reads your current attempt and asks a question instead of giving the answer. You can also paste your own working code: the AI writes tests for it, keeps only the ones your code passes, then breaks it.
  • 📈 Progress. Mastery per bug type, a drill-history heatmap and a count of bugs you hit in your own code. The next drill targets whichever is weakest, so it trains your blind spots.

How we built it

Python + Streamlit for the app. Gemini Flash-Lite (Google AI Studio, JSON mode, minimal thinking for ~3-second answers) powers four jobs: diagnosis, bug injection, test writing and Socratic coaching, with Groq (Llama 3.3 70B) as an automatic fallback. Every piece of AI output that makes a claim about code goes through a sandbox: an isolated Python subprocess with a timeout and CPU/memory caps that runs each test separately and reports structured results, including the actual value on a failed assert. A verifier accepts an AI-generated bug only if it parses, keeps the same functions, changes at most a few lines, doesn't hang, and fails at least one test; rejected bugs go back to the model with the reason, up to three tries. A hand-verified offline bug bank (19 bugs across 10 categories) means the gym never dead-ends if the API is down. 48 pytest tests cover the sandbox, every seed bug, the verifier and the retry loop (with the LLM mocked).

How we used AI

  • Gemini for diagnosis (T=0.2), bug injection with hints and explanations (T=0.8), test generation (T=0.2) and live Socratic nudges (T=0.5), all with strict JSON output.
  • Groq / Llama 3.3 70B as a fallback provider.
  • Design principle: the model proposes, the sandbox disposes. No AI-written bug, test or fix reaches the student without being executed.
  • I also used an AI coding assistant (Claude) while building; all code was written during the hackathon and reviewed and tested by me.

Challenges we ran into

  • LLMs write convincing but wrong bugs. Early mutants often still passed every test (no real bug) or changed half the function. The verify-and-retry loop with the rejection reason fed back to the model fixed most of this.
  • AI-written tests can be wrong too. For pasted code, tests are run against the student's working code first and any that fail are dropped.
  • Showing "got X" on a failed assert without calling the function twice (which corrupted results for stateful bugs like mutable defaults). I rewrote each assert's AST to capture the left-hand value first.
  • Making it reliable for a live demo: the first model I tried timed out on the free tier, so I added per-model timeouts with automatic fallback to the next model, a Groq fallback, an in-app AI health check, and the offline bug bank.

Accomplishments that we're proud of

A full loop that a judge can try in two minutes: break → fail → hint → fix → green → explanation → next adaptive drill. And every AI claim is checked by running code.

What we learned

Prompting is half the work; verification is the other half. Grounding an LLM with executable checks turned it from "usually right" into something I'd trust in front of a beginner.

What's next for GlitchGym

Pyodide so code runs in the browser; JavaScript and Java; instructor mode that turns a lab's starter code into a set of verified bug drills for the class; spaced repetition; and a VS Code extension that offers a drill straight from a real traceback.

Try it

Built With

Share this project:

Updates

Submission history