Spendlint

Inspiration

Every LLM observability tool I looked at works the same way: it tells you after a cost spike which deploy caused it. The bill has already arrived.

The actual cause is almost always a one-line diff. A model swap. A retry loop added around an existing call. A max_tokens bump. These changes are invisible at review time because nobody has tooling that reads a diff and says "this costs $14/day more." That gap is what Spendlint fills.

What it does

Spendlint is a pre-merge LLM cost gate for GitLab. When a developer opens a merge request, spendlint's webhook fires. A Gemini 2.5 Pro agent fetches the diff, classifies each LLM-touching change (model swap, volume multiplier, token cap change, new call site), projects the dollar delta against real historical traffic from the ledger, and posts a verdict comment on the MR - automatically, before any human reviewer touches it.

The comment shows the math: baseline $/day, projected $/day, delta, and the assumptions used. PASS / WARN / BLOCK verdict based on the delta magnitude.

How we built it

• Agent loop: Gemini 2.5 Pro on Vertex AI via the Go google.golang.org/genai SDK. The agent calls four tools in sequence: get the diff, analyze cost impact, compose the comment, post it.
• Diff classifier: regex + heuristic parser that identifies change types (model_swap, volume_added, max_tokens_change, call_added/removed) from unified diff hunks.
• Cost projector: projects delta using calls_per_day * (new_rate - old_rate) against per-label traffic averages stored in a SQLite ledger.
• Ledger + recorder: a pkg/client library teams embed at each LLM call site to log token counts and cost with a stable label. The label is the join key between code and traffic.
• GitLab integration: GitLab MCP server (local dev, OAuth via mcp-remote) and REST API (Cloud Run, PAT-based). MR webhook triggers the review goroutine.
• Deployment: Cloud Run with a GCS volume mount for persistent SQLite. Min instances = 1 so the async review goroutine survives past the 202 response.
• Language: Go 1.25, no CGO, pure-Go SQLite via modernc.org/sqlite.

Challenges we ran into

Connecting a code change to its historical traffic is the hard part. The diff tells you a line changed; the ledger knows traffic by call-site label. Bridging them required a stable labeling convention and a diff-to-label resolver that uses surrounding context.

The other challenge was the GitLab MCP server auth. PATs don't work - OAuth only, via the mcp-remote npm bridge. Debugging that took a day. The fix was a one-line account config change (setting a default Duo namespace), not a code change, which made it harder to find.

Accomplishments that we're proud of

The cost projection actually works on real diffs. A one-line model swap in Go - const defaultModel = "gemini-2.5-flash" - gets classified correctly, the dollar delta is computed against real traffic, and the comment is posted to the MR in under a minute. The entire path from webhook to comment is end-to-end automated with no human in the loop.

What we learned

The shift-left framing is the right one for cost. Teams instrument observability after a spike. But the cause is always in a diff that shipped days earlier. Catching it at review time, with a concrete dollar number attached, changes the conversation from "why did the bill go up" to "do we actually want to merge this."

What's next for Spendlint

• Tighter diff-to-label resolution using GitLab MCP semantic_code_search.
• Blocking merges via GitLab approval rules when delta exceeds a configurable threshold.
• Multi-provider pricing table updates (new models ship constantly).
• A richer dashboard with per-label trend charts and MR review history.

Built With

• gemini-2.5-pro
• gitlab-mcp-server
• gitlab-rest-api
• go-1.25
• google-cloud
• google-cloud-run
• sqlite
• vertex-ai