Inspiration
Two failures started this project. First, I asked a local LLM to review my git diff and return JSON. One day it added a trailing comma, my parser crashed, and my pre-commit check silently passed. The exact bug it existed to catch went straight through. Second, I looked at cloud code review tools and realized the deal on offer: mail your unreleased code to someone else's server and trust their retention policy.
So the goal became a code reviewer with guarantees instead of hopes: fully local, structurally reliable, and fast enough that you'd actually leave it on.
What it does
mygit is a C++20 CLI wrapper around Git. Before every commit or push, it extracts your staged diff, retrieves semantically related code from your own repository, and runs a local LLM review. Issues come back with severities (critical, high, medium, low). A critical issue blocks the commit; --force-ai overrides it, because a tool that can't be overruled gets uninstalled. It also generates Conventional Commit messages from your diff, with a [Y/n/e] prompt to accept or edit. Every verdict is stored in SQLite and browsable via mygit history. No cloud, no API keys, no network calls.
How we built it
Reliability first. Instead of prompt engineering and hoping, I attached a GBNF grammar to llama.cpp's sampler. Tokens that would violate the JSON schema are masked out at decoding time, so malformed output is mathematically impossible.
Native Git, no shell. Early versions parsed git diff text through popen. That's brittle across platforms, so I ripped it out and embedded libgit2, reading diffs straight from the object database with RAII wrappers around every C handle. This also enabled rename detection, so a moved file no longer looks like 400 deleted lines plus 400 new ones.
Speed. A resident daemon holds the model in memory and serves the CLI over local HTTP. On top of that: prefix KV caching (the system prompt is evaluated once and reused across commands), per-file batched reviews with an aggregation pass, a content-addressed SQLite cache so unchanged diffs never re-run inference, and an async background writer so the terminal returns the instant the verdict does.
Context. The RAG pipeline parses the repo with Tree-sitter into logical code units, embeds them with an ONNX Runtime model through a from-scratch byte-level BPE tokenizer, and indexes them in FAISS. Indexing is incremental by content hash, and retrieval prepends the top-k relevant units to the prompt. If no embedding model is present, everything degrades gracefully to plain diff review.
Challenges we ran into
The hardest fight was building ONNX Runtime with CUDA 13 on Windows with a preview MSVC toolset. That produced a vcpkg overlay port with pinned CUDA architectures, /Zc:preprocessor for CUDA's CCCL headers, a one-line extern template patch for an MSVC optimizer bug, disabled contrib ops to dodge OOM-inducing flash-attention kernels, and a workaround for Windows' 260-character path limit. In 2026.
The subtler challenge was latency psychology: a reviewer that adds ten seconds to every commit gets removed within a week. Most of the architecture exists to make the review feel free.
Accomplishments that we're proud of
Zero JSON parse errors since the grammar went in, by construction rather than by luck. A complete local RAG stack (parser, tokenizer, embedder, vector store) in C++ with graceful fallback at every layer. Cold starts eliminated through the daemon plus three layers of caching. And the tool was its own test arena: mygit reviewed every commit made to mygit, blocking me with bugs in the code that catches bugs.
What we learned
Constraints beat instructions: the grammar solved in one move what prompt engineering never fully could. Native APIs beat shelling out, every time. Graceful degradation is a feature you design, not a property you assume. And dogfooding is the fastest feedback loop there is.
What's next for AI_Git_Reviewer_local
Parallel per-file review batches, task-specific model routing (a small model for commit messages, a larger one for review), quantization benchmarking across INT8/INT4/FP16, and auto-fix: generating unified patches for detected issues, previewed and applied on approval.
Log in or sign up for Devpost to join the conversation.