TestPilot

An AI-powered QA platform and CLI that automatically detects, audits, and safely fixes bugs in your code using Gemini AI.

Comment

Inspiration:

I spent way too much time tracking down small typos, reading stack traces, and fixing minor bugs that broke my build. Debugging takes away time that should be spent building features. Commercial code audit tools are also very expensive. I wanted something fast, safe, and affordable that works as a helpful assistant directly in my terminal and web browser.

What it does:

TestPilot AI is a dual-interface platform that automates QA and debugging:

  • Web Dashboard: Generates free repository audit reports categorized into Optional, Recommended, and Compulsory fixes using your own Gemini API key.
  • Developer CLI: Inspects local Git workspace changes (git diff), runs background Playwright tests when code breaks, and automatically proposes verified code fixes.

How we built it:

  • Web Dashboard: Built with React, Vite, and Tailwind CSS.
  • Developer CLI: Built with Node.js, TypeScript, Commander.js, and @clack/prompts.
  • Backend API: Node.js, Express, and MongoDB for handling context parsing and encrypted key storage.
  • AI & Testing: Google Gemini API for code analysis and Playwright for automated test execution.

Challenges we ran into:

  • Preventing Broken Code: Making sure the AI doesn't ruin working code. I solved this by adding a strict Suggest → Review → Approve → Apply → Verify workflow with an automatic Rollback system if a fix fails.
  • Keeping Prompts Fast & Cheap: Passing full repositories to AI is slow and costly. I limited the AI prompt to only changed files from git diff and their matching Playwright tests.

Accomplishments that we're proud of:

  • Successfully building individually an end-to-end local auto-debugging workflow that detects bugs, proposes fixes, applies them, and re-verifies tests automatically.
  • Keeping the system fully cost-efficient and secure by letting users use their own encrypted Gemini API keys.

What we learned:

  • How to build interactive terminal CLI tools using Node.js and TypeScript.
  • How to run and parse Playwright tests programmatically inside a CLI tool.
  • How to pass minimal Git context to LLMs for fast and accurate code fixes.
  • How to read commands and run a CLI tool.

What's next for TestPilot:

  • Multi-Developer & Team Workflow: Automatically scan and detect bugs introduced by team members in shared repositories, instantly sending automated fix suggestions via notifications to frontend, backend, or DevOps teams.
  • Subscription & Team Plans: Monetize the platform by offering paid team tiers with centralized dashboard reporting, shared team configurations, and custom notification integrations (like Slack and Discord).
  • GitHub Actions & CI/CD Integration: Automatically run TestPilot scans on incoming Pull Requests to catch and fix regressions before code is merged.
  • VS Code Extension: Launch a dedicated extension to provide real-time inline AI suggestions directly inside the code editor.

Built With

Share this project:

Updates

Submission history