Is my repo compromised

Paste a GitHub repo, get a polished supply-chain report: real 2026 attack IOCs + OSV.dev CVEs, severity-sorted, with deterministic fix diffs the LLM never touches.

Comment

Inspiration

2026 has been a bad year for open-source supply chains. In the last few months alone, a worm campaign compromised 160+ npm and PyPI packages including TanStack and Mistral AI, a compromised Axios release shipped a North Korea-attributed RAT, and a poisoned LiteLLM package used a .pth file trick that bypasses pip's normal hash verification entirely. Most of these attacks are technically detectable — the malicious package/version pairs are public knowledge — but nobody checks, because npm audit output is an unreadable wall of CVE IDs that developers learn to ignore. I wanted to close that gap: paste a repo, get a plain-English answer to "am I affected," not a report you need a security background to parse.

What it does

Paste a public GitHub repo URL and the tool detects its dependency manifest ( package.json or requirements.txt ), then checks every dependency against two sources: a curated list of knownmalicious package/version pairs from active 2026 attack campaigns, and OSV.dev's live vulnerability database. Anything that matches gets explained in plain English — what it does if exploited, how severe it is, and the exact safe version to upgrade to, shown as a ready-to-copy fix diff. Clean repos get a clean verdict in under a second. The whole scan is cached by commit SHA, so re-scanning an unchanged repo costs nothing

How I built it

Backend is FastAPI. The scan pipeline resolves the repo's default branch and commit SHA through the GitHub API, pulls the manifest from raw.githubusercontent.com (unauthenticated and unmetered, so it doesn't burn API quota), and parses out dependency name/version pairs. Those get checked against a local curated IOC list first — instant, no API call — and anything unmatched goes to OSV.dev in a single batch query. Only confirmed hits ever reach the LLM: a single Groq call using llama-3.3-70b-versatile that explains and prioritizes the findings, with a system prompt that explicitly forbids it from inventing or re-grading anything not in the input data. The fix diffs aren't LLM output at all — they're computed deterministically in the backend from the confirmed safe version, so the one piece of information a user might actually paste into their codebase is never something the model made up. Frontend is a dependency-free single page with staged scan progress so you can watch the pipeline work instead of staring at a spinner.

Challenges I ran into

The first real problem was rate limits, stacked two deep. Unauthenticated GitHub API calls cap at 60/ hour total — shared across every user — so a GitHub personal access token was non-negotiable before writing any scanning code. Second, Groq's free tier throttles llama-4-maverick harder than its other models, and this task (summarizing structured vulnerability data, not novel reasoning) didn't need Maverick's extra capacity anyway, so I switched to llama-3.3-70b-versatile for roughly 4x the token budget at no cost. The bigger challenge was trust, not throughput. An LLM asked "is this package vulnerable" will confidently hallucinate CVEs if you let it reason freely — that's a known failure mode, and shipping that in a security tool would be worse than shipping nothing. The fix was architectural: the LLM never sees raw dependency lists, only pre-confirmed matches from OSV.dev and the curated IOC list, and its prompt explicitly forbids inventing or re-scoring anything. Fix diffs are computed with plain string logic in the backend, never generated by the model. I also had to guard against my own scan being unreliable on a bad day: OSV.dev being slow or down, Groq timing out, or a manifest with 200+ dependencies blowing the token budget in one request. Every one of those degrades gracefully instead of failing the whole request over one flaky upstream call

Accomplishments that I'm proud of

151 passing tests with every upstream service (GitHub, OSV.dev, Groq) mocked, covering the happy path, every error state, the cache-hit path, and both LLM failure modes. I'm also proud of the hallucination-proofing specifically — it would have been much faster to just ask the LLM "is this repo compromised" and take whatever it said, and resisting that shortcut is the difference between a demo and a tool I'd actually trust.

What I learned

How to design around rate limits as a first-class constraint instead of an afterthought. I also learned a lot about how real 2026 supply chain attacks actually work by researching the campaigns this tool is built to catch, which made the "why this matters" case a lot more concrete than a generic "security is important" pitch.

What's next for Is my repo compromised

Support for more ecosystems beyond npm and PyPI (Cargo, Go modules), monorepo-aware scanning that checks subdirectories instead of just the repo root, and lockfile parsing for exact resolved versions rather than manifest ranges. Longer term, I'd like to go from "shows you the fix" to "opens the fix as a real pull request" — I deliberately scoped that out of this build since it needs OAuth and write access I didn't have time to build safely, but it's the natural next step once the detection side is solid.

Built With

Share this project:

Updates

Submission history