Inspiration
Every team I've worked with had the same two problems: PR reviews were either slow (waiting on a senior dev) or shallow ("LGTM 👍" with zero actual reading), and nobody was watching the CI/CD pipeline at all — unpinned actions, leaked secrets, tampered lockfiles just sat there. Tools existed for each problem separately, but reviewers like Greptile don't touch pipeline security, and scanners don't understand your code. I wanted one platform that reviews the code and guards the pipeline it ships through — so I built LGTM (Looks Good To Meow).
What it does
LGTM is a GitHub app that does two things on every PR:
- AI code review — six specialist lenses (bugs, security, performance, readability, best practices, docs) that produce tiered, evidence-backed findings and can chat about the diff.
- CI/CD security — 36 deterministic detectors across GitHub Actions, GitLab CI, CircleCI, Azure Pipelines, Docker, and IaC, with three enforcement gates: inline comments, a merge-blocking Check Run, and a runtime Watchdog action that halts a compromised pipeline mid-run.
On top of that: CycloneDX SBOM generation across 9 ecosystems, a 0–100 Posture Score with trends and a public README badge, org-level RBAC and shared policies, a YAML DSL for writing custom detectors, and a CLI.
How I built it
- Stack: Node.js + TypeScript API and workers, React + Vite dashboard, MongoDB, Upstash Redis + QStash for queues, deployed on Fly.io.
- Review pipeline: originally six separate LLM agents. I rebuilt it into two structured-output LLM calls (Correctness + Style) plus a batched adversarial verifier and a synthesizer — same depth, 5–10× cheaper and ~4× faster.
- Context engine: tree-sitter indexes symbols across 12 languages, cached per repo and refreshed on push. Context assembly dropped from ~10s to ~1s, and GitHub API usage per review fell from 20–40 calls to 3–5.
- Hallucination filter: every finding must quote the offending code verbatim; anything the verifier can't ground in the actual diff is silently dropped.
Challenges I faced
- LLM cost vs. depth. Six agents per PR gave great coverage but burned money. Merging them into batched structured-output calls without losing quality took many eval rounds — the adversarial verifier was the key unlock.
- Trusting AI in a security product. A reviewer that hallucinates a vulnerability is worse than none. The quote-verbatim-or-drop rule cost me some recall but bought credibility.
- Prompt injection. PR diffs and comments are attacker-controlled input to the LLM. Everything user-controlled gets wrapped in explicit
<untrusted>boundaries before it touches a prompt. - Enforcement, not just reporting. Making the Watchdog actually halt a running pipeline (with mintable scoped tokens) was much harder than posting a comment — but it's the difference between a linter and a gate.
What I learned
- Structured outputs + batching beat "one agent per concern" on cost, latency, and consistency.
- Deterministic detectors and LLM judgment complement each other — regex-style rules catch what AI misses, and vice versa.
- Shipping a real product (billing, quotas, RBAC, refund policy) is 3× the work of building the core tech — and where most of the learning happened.
What's next
Deeper org analytics (MTTR, false-positive leaderboards), a cross-org detector marketplace, and expanding the custom-detector DSL.
Log in or sign up for Devpost to join the conversation.