ExplainIt

Don't just vibe-code: actually understand it. ExplainIt is an MCP-powered agent that watches AI edits in your active workspace and tests whether you know the code before you blindly hit accept.

Comment

The inspiration for ExplainIt

I love the speed of modern AI-assisted development. With Copilot, Cursor, Codex, and other agentic tools, you can spin up whole features in minutes just by describing what you want. But recently, I noticed a really bad habit creeping into my workflow, and I know it's not just me.

I call it the “looks good to me” reflex. An AI agent spits out a 40-line diff, I glance at it for three seconds, see that it compiles without red squiggles, and hit save or commit. Then, an hour later, everything falls apart because an empty array caused an unhandled edge case, a state transition broke, or an async call was completely unhandled. I spent 30 minutes debugging code that I technically "wrote," but had zero actual comprehension of.

The reality is that while "vibe-coding" is intoxicating for a lot of developers, passive code acceptance is super dangerous. Most AI tools act like eager yes-men and write code for you, but they don't help you understand what just happened.

I wanted to flip that dynamic on its head. Instead of the AI being the one that writes and explains, what if the AI acted like a mischievous retro game-show host that immediately halts your momentum and quizzes you? What if you had to prove you were paying attention before you earned the right to feel finished with a change?

What ExplainIt does

ExplainIt is a retro game-show host and code comprehension companion that lives directly inside VS Code:

  • Watches your workspace seamlessly: Automatically notices when a meaningful code change is saved in your project, eliminating manual copy-pasting.
  • Pinpoints the newest edit: Instead of getting distracted by an old 300-line git diff, ExplainIt extracts the discrete latest insertion, deletion, or modification signal.
  • Hosts an interactive comprehension quiz: An avatar-led retro game-show webview pops open with a 4-question challenge tailored specifically to the logic, data flow, and edge cases of the code that just changed.
  • Blends question formats: Keeps you on your toes by mixing single-choice recognition questions with free-form explanation questions.
  • Instant code-grounded feedback: Evaluates your answers on the fly—giving credit for correct technical reasoning, catching when you honestly admit "I don't know," and explaining the underlying edge case.
  • Generates a Trust Receipt: Summarizes the inspected workspace context, which Model Context Protocol (MCP) tools were called, why the questions were chosen, and what areas of the code you should double-check.
  • In-editor chat companion (@explainit): Lets you route prompts directly to the host inside VS Code Chat for quick commentary on what to watch out for as you build.

Basically, I created an in-IDE comprehension guardrail that forces vibe-coders to actually master the code they'd like to ship, which I thought was a clever idea given the theme of this hackathon.

How I built it

1. VS Code Extension & Retro Webview (extension/)

The frontend is a custom TypeScript VS Code extension. It registers save listeners (onDidSaveTextDocument) and active document trackers to capture the exact delta of the developer's newest edit. When triggered, it reveals a standalone Webview with a pixel-inspired retro game-show aesthetic—complete with a custom avatar, arcade typography accents, sound/buzzer vibes, instant answer locking, and interactive score reveals.

2. Local ExplainIt Service (service/src/server.ts)

A lightweight Node.js HTTP companion service running locally on port 4242. It decouples the extension UI from heavy AI orchestration and API keys. The service exposes /quiz, /context, /observe, and /health endpoints, managing the agent prompt loop, call budgets, and fallback mechanisms.

3. Model Context Protocol (MCP) Architecture (service/src/mcp/)

To ground the quiz strictly in safe, local project context without dumping whole repositories into LLM context windows, I built an MCP server using the @modelcontextprotocol/sdk. Running over a stdio child process, it exposes read-only tools to the agent:

  • get_focused_change: Fetches the bounded Git diff and generates a fingerprint for the target file.
  • get_nearby_context: Retrieves surrounding code baseline via git show HEAD:<file> (clamped to 8,000 characters).
  • get_workspace_metadata: Exposes safe project metadata without touching credentials or arbitrary paths.

4. Safety & Boundary Filtering (service/src/workspace/safety.ts)

A strict boundary layer ensures the MCP server blocks sensitive files like .env, .env.local, credentials.json, or secrets.json, preventing secrets or arbitrary directory traversal from ever reaching the agent or model prompts.

5. Multi-Provider LLM & Self-Repair Schema Engine (service/src/agent.ts)

ExplainIt supports multiple model providers (Gemini gemini-3.8-flash, Groq openai/gpt-oss-20b, OpenAI gpt-4o-mini, and Nvidia). Quizzes are generated using JSON mode validated with Zod against a strict 4-question contract (either 2 single-choice + 2 free-form or 3 single-choice + 1 free-form). If a model produces malformed JSON or invalid question distributions, an automated single-repair loop prompts the model to self-correct before the quiz ever renders.

6. Standalone VSIX & Single-Window Workflow

To make ExplainIt feel like an actual product rather than an awkward hackathon demo, I packaged the extension into a standalone .vsix bundle. Developers can install it directly into their everyday VS Code window without needing to stay tethered to a separate debugging host (F5).

Challenges I ran into

  • Stale Git Diffs vs. Discrete Edit Signals: Initially, the agent was quizzing the developer on the entire accumulated Git diff of a dirty file. If you had 50 lines of uncommitted boilerplate and just changed one line in a helper function, the quiz would ask about the boilerplate. I had to build a document snapshot diffing system in extension.ts that calculates the newest discrete edit (lines inserted/deleted) and forces the agent prompt to prioritize that latest action.
  • Focus Shifts with Copilot/Chat Saves: When testing with GitHub Copilot, saving a file from an AI chat view often fired save events while the editor window wasn't technically active. The extension originally assumed vscode.window.activeTextEditor was always present. I had to refactor the event pipeline to track document URIs directly from the save event itself.
  • Provider Idiosyncrasies with JSON Mode: When adding support for Groq and reasoning models, I hit unexpected provider errors—Groq rejected plain-text prompt observation when JSON mode was enabled, and counted hidden reasoning tokens toward completion budgets. I separated the plain-text host commentary pipeline from the structured quiz pipeline and tuned completion token limits.
  • Balancing Extension Responsiveness with Fallbacks: Network latency or a stopped local service could make the command feel completely broken. I implemented a 2-second timeout with an instant loading panel and an automatic fallback to fixture mode, ensuring the user always gets immediate visual feedback and never feels stuck.
  • Free-Form Answer Grading: Scoring free-form answers in real-time without burning another expensive round-trip model call was tricky. I engineered an in-memory heuristic grading engine in quiz-panel.ts that combines stop-word filtering, word stemming, concept canonicalization, and an uncertainty detector (catching phrases like "idk" or "not sure") to grade responses accurately and instantly.

Accomplishments that I'm proud of

  • A Genuinely Fun, Cohesive UX: The retro game-show identity completely changes the mood. Instead of feeling like a boring linter or annoying pop-up, getting challenged by the avatar with immediate feedback and a reveal score feels like a quick mini-game during your coding flow.
  • The Trust Receipt: Judges and developers love transparency. The Trust Receipt provides tangible proof of what the agent did—highlighting exactly which MCP tools were used, which lines were read, what edge case was tested, and where to look next.
  • True MCP Integration: I didn't just prompt an LLM; I implemented the Model Context Protocol from scratch with real client-server stdio communication, strict tool isolation, and workspace safety filters.
  • Rock-Solid Fallbacks: ExplainIt never crashes or hangs silently. Whether an API key is missing, a service is offline, or a diff is empty, the UI explains what's happening gracefully.
  • One-Window Polish: Packaging the project into an installable .vsix means you can use ExplainIt in your actual daily coding projects, not just inside an isolated dev sandbox.

What I learned

  • MCP is about discoverability and tool boundaries, not magical memory: Before this hackathon, I thought MCP was some sort of persistent memory layer. Building ExplainIt taught me that MCP is fundamentally a clean, standardized contract for tool discovery and execution. It separates how an agent inspects the world from how the agent reasons.
  • Friction can be a feature when it’s engaging: Developer tools are usually obsessed with removing every possible second of friction. But when you remove all friction from code generation, developers stop thinking. Introducing intentional, game-like friction actually saves hours of downstream debugging.
  • Event deltas beat whole-state snapshots: Passing an entire repository or file to an agent leads to generic, wandering questions. Giving the agent the narrowest possible event delta (the 2 lines just deleted + surrounding 10 lines) produces dramatically sharper, high-signal questions.
  • Contract-first agent architecture: Defining strict Zod schemas with repair retry loops is 100x more reliable than hoping a system prompt returns well-behaved markdown.

What's next for ExplainIt

  • Streaks, Badges, and Leaderboards: Adding persistent comprehension stats, daily streaks, and arcade-style badges ("Edge Case Hunter", "Zero-Bug Hero") to gamify daily coding sessions even further.
  • Direct AI Diff Interception: Exploring deeper hooks into extensions like GitHub Copilot and Cursor so the host can chime in the microsecond an agent generates code, before the developer even presses save.
  • Multi-File Context Reasoning: Extending the MCP server to inspect cross-file dependencies and imports when a change in one file affects contracts in another.
  • Customizable Host Personalities: Allowing developers to pick their game-show host persona—from encouraging retro arcade host to sarcastic tech lead grilling your PR.
  • VS Code Marketplace Release: Polishing the packaging and publishing ExplainIt to the official VS Code Extension Marketplace.

Built With

Share this project:

Updates

Submission history