CodeAudit AI — Project Story

Inspiration

Every developer has shipped code they weren't proud of — not because they didn't care, but because proper code review felt out of reach. Setting up SonarQube requires a CI pipeline. Hiring a senior engineer costs money. Asking a colleague takes time nobody has.

We kept seeing the same pattern: developers skipping the review entirely and hoping for the best. CodeAudit AI started as a simple question:

What if the review could just happen automatically, the moment you paste a link?

What It Does

CodeAudit AI analyzes any public GitHub repository and returns a structured quality report in under 30 seconds. The output covers:

Report Section	Description
Quality Score	A numeric score from 0 to 10 reflecting overall code health
Architecture Summary	AI-generated description of structure and design patterns
Security Flags	Vulnerabilities ranked by severity: `high`, `medium`, `low`
Recommendations	Prioritized, actionable improvements
Generated README	A full Markdown README produced from the codebase

No account. No configuration. No setup. Just a URL and a report.

How We Built It

The system follows a five-stage pipeline:

URL → Parse → File Tree → Select Top Files → Prompt Gemini → Report

Stack:

Frontend — React 18 + Vite, deployed on Vercel
Backend — Node.js Vercel Serverless Function (/api/analyze.js)
AI — Google Gemini 1.5 Flash with responseMimeType: application/json
Data — GitHub REST API v3 (file tree + raw file contents)

The backend orchestrates the full pipeline in a single function: parse the URL, fetch the file tree, select files, build the prompt, call Gemini, return structured JSON.

Challenges We Ran Into

1. The File Selection Problem

Repositories can have thousands of files, but Gemini has context limits. We needed to select a fixed budget of 10 files that maximizes the information available for analysis. We assigned each file a priority score based on its tier:

Tier 1 (score 3) — manifests, README, dependency files
Tier 2 (score 2) — entry points like index.js, main.py, app.py
Tier 3 (score 1) — source files, sorted by size descending

Files are ranked by priority, with ties broken by file size, and the top 10 are selected.

2. Consistent JSON Output from Gemini

Early prompt versions occasionally returned markdown fences or inline explanation text, breaking JSON.parse(). Setting responseMimeType: application/json and temperature: 0.3 solved it — lower temperature reduces creative variation and forces adherence to the output schema.

Accomplishments We're Proud Of

The pipeline works on repositories of all sizes, including large-scale codebases like vercel/next.js and facebook/react
Quality scores are calibrated — a well-documented, tested repo consistently scores 8 or above, while a bare prototype scores 4 or below
The full end-to-end analysis completes in under 30 seconds for most repositories
The experience is genuinely frictionless: zero sign-up, zero configuration

What We Learned

Prompt engineering is real engineering. The difference between a prompt that returns inconsistent output and one that reliably produces a structured, useful analysis is significant — and it requires the same rigor as writing production code.

We also discovered that the file selection problem is more interesting than it looks. Choosing which 10 files to send to an AI is effectively a lossy compression problem: how do you maximize the information available for analysis within a fixed budget? In practice, manifest files and entry points carry disproportionately high signal — a finding that shaped the entire algorithm.

What's Next for CodeAudit AI

Private repo support via GitHub OAuth
Shareable report links so teams can review audits together
Diff mode — re-analyze after changes and surface what improved
IDE extensions for VS Code and JetBrains
Team dashboard for tracking quality score trends across repositories over time

The long-term goal: make professional-grade code review accessible to every developer, regardless of team size or budget.