RepoPilot

RepoPilot checks any public GitHub repo’s security and health in 2 seconds: a score out of 100, evidence for every finding, and a ranked list of what to fix first. Free, no AI, never runs code.

Comment

Inspiration

Every modern app is built on open-source code written by strangers. Before you add a dependency, inherit a codebase or launch a project, you need answers fast: Is it tested? Does its CI run risky code? Has someone committed a secret? In practice that means hours of clicking through folders, a heavyweight platform that needs setup and access, or an AI chat answer that can't show its evidence.

I wanted a tool that answers in seconds, shows proof for every claim, and never touches the code it's judging.

What it does

Paste any public GitHub URL and RepoPilot:

  1. Fetches a bounded sample of the repository through GitHub's API (4 API requests, strict time and size limits). The code is never cloned, installed or run.
  2. Detects the stack (languages, frameworks, test tools, CI).
  3. Runs 30 deterministic rules across security, testing, maintainability and project hygiene: committed secrets (always redacted), dangerous pull_request_target workflows, unpinned third-party actions, missing security policies, missing tests, and more.
  4. Returns a 0–100 score where every finding has a rule ID, the evidence (file and line), why it matters and how to fix it.
  5. Ranks every gap in a "Fix next" plan by exactly how many points fixing it adds, and for 17 rules generates a fix-it kit: ready-made files (a first test, a least-privilege CI workflow, a security policy) written for that repository's stack. Reports export as Markdown and JSON, and each one has a README badge and a link preview that shows the real score.

How I built it

Next.js App Router + strict TypeScript, styled with Tailwind, deployed on Vercel's free tier. Running cost: ₹0. A pure analysis core. The engine takes a snapshot of facts and returns a report. It knows nothing about HTTP, so every rule can be unit-tested against a fake GitHub. Zod validates every report against a versioned schema. Security first: user input is parsed into owner/repo and only two fixed GitHub origins are ever contacted (no arbitrary URL fetching). Redirects are checked manually, every request has a timeout and a byte cap, and the optional GitHub token lives only on the server. Potential secrets are redacted inside the scanner, before anything is stored or shown. Live progress streams to the browser as NDJSON, with a per-instance rate limiter, a concurrency gate and a 10-minute cache. ~260 automated tests (Vitest) run in GitHub Actions on every push, alongside lint, format, typecheck and build.

Challenges I ran into

My first version was wrong a third of the time. I scanned 26 well-known repositories and checked every finding by hand: precision was only 66%. Most false positives came from things it couldn't see: organization-wide policy files kept in {owner}/.github, custom test runners like ./runtest, and tools invoked from npm scripts. I fixed each root cause, added a regression test for each, and re-measured: 91%. Honest evaluation. A score measured on the repos you tuned on isn't a real result. So I picked 15 new repositories before scanning them: precision was 81.8%, with 100% agreement with GitHub's community profile API and OpenSSF Scorecard on every overlapping check. That's the number I stand behind. The hold-out set also exposed new edge cases (symlinked READMEs, public PGP keys flagged as secrets), which I fixed and then regression-tested against the original study set. Rate limits. Unauthenticated GitHub allows 60 requests an hour, so a scan had to fit in 4 API calls. File contents come from raw.githubusercontent.com, pinned to the commit SHA. A new framework version. This Next.js release changed APIs, so I worked from its bundled docs rather than old tutorials.

Accomplishments that I am proud of

It's live and it works. Real scans of Go, Rust and Python repositories take about 1.2–1.6 seconds in production, on a ₹0 budget. Measured accuracy instead of claimed accuracy. I took precision from 66% to 91% on the study set, and published an 81.8% hold-out result with 100% agreement with two independent sources. The study is reproducible from scripts in the repo. Fixes, not just findings. Tests prove that applying a fix-it kit makes its check pass, and that the generated CI workflow passes RepoPilot's own security checks. Safe by design. It never executes analyzed code, never fetches user-controlled URLs, and redacts every matched secret. Engineering quality. 260 tests, strict TypeScript, green CI on every push, and RepoPilot scores 100 (band A) on its own repository. Honest product. Every limitation is shown on the report, and nothing is claimed that wasn't implemented and tested.

What I learned

Measure before you claim. A deterministic tool can still be confidently wrong, and a hold-out set is the only honest test. Explainability is a feature. Rules plus evidence beat a black box when people have to act on the result. Small, pure cores are easy to trust. Separating the engine from the web layer made 260 tests possible and refactors safe. Security is mostly about saying no: to arbitrary URLs, unbounded downloads and running untrusted code.

What's next for RepoPilot

Score history and comparisons, private repositories for teams, and a larger, independently reviewed accuracy study. The paid tiers are proposals I'm still validating; nothing is for sale yet.

Built With

Share this project:

Updates

Submission history