Lou | Devpost

Inspiration

Every team has a backlog item that says "fix the slow endpoint." Nobody fixes it, because nobody can prove it's worth fixing. Technical debt is invisible until it's an outage.

We kept noticing that the tools around us had the same blind spot. Linters find style problems. Profilers find slow code after you ship it. CI tells you tests passed. None of them answer the question an engineering manager actually asks: which of these problems is going to cost us the most, and can you just fix it?

So we built Lou — named in the spirit of Claude and Shannon, after a person rather than an acronym. Lou reads your repo, measures it, decides what's worth acting on, and only then asks for permission to act.

What it does

Lou watches a repository for a performance regression, proves it with real measurements, asks Gemini to write the fix, and verifies the fix before anyone looks at it.

The pipeline is seven deterministic stages: validate → initialize → inspect → select → verify → decide → finalize.

Inspect — builds a NetworkX graph of the repo (files, functions, classes, tests, endpoints, database tables) and diffs two commits to find what actually changed.
Select — picks the load workloads that exercise the changed code. No point benchmarking untouched paths.
Verify — spins up Docker and runs k6 against baseline and candidate, five repetitions each, counting SQL queries and latency.
Decide — scores technical debt as principal + interest, where principal comes from complexity, coverage deficit, and patch size, and interest is weighted by churn, graph centrality, runtime impact, and path criticality.
Act — Gemini gets a bounded evidence bundle and returns a corrected file. Lou builds the diff itself.

The demo case is an N+1 query. The good version runs 2 queries. The regression runs 51. Lou catches it, Gemini fixes it, and Lou re-runs the same workload to confirm the fix actually worked.

Critically, Lou has an autonomy ladder: A0 report → A1 recommend → A2 generate patch → A3 open PR. It only climbs when the evidence supports it. Weak evidence means Lou tells you something and stops.

How we built it

Python 3.14, Pydantic v2 with frozen contracts and extra="forbid", strict mypy, ruff. FastAPI and a Typer/Rich CLI. PostgreSQL with Alembic for run history. Docker and k6 for the measurement harness. NetworkX for the repository graph. Google Gemini via the google-genai SDK for diagnosis and repair.

Two architectural decisions did most of the work for us:

Protocol-based ports. AgentProvider, Verifier, RepositoryIntelligencePort, WorkloadSelectionPort — four people built four subsystems against interfaces, in parallel, and they composed on first merge.

Never trust the model with a diff. Gemini returns the complete corrected source of one file. Our code runs difflib to produce the patch, validates the target path can't escape the repo root, and confirms the file was actually in the evidence bundle. A model that hallucinates can produce a bad file. It cannot produce a bad diff, because it never touches the diff.

Challenges we ran into

The safety bug that inverted. Our autonomy gate took the average of debt confidence and remediation confidence. Which meant that when remediation evidence was missing, confidence went up — withholding evidence promoted Lou from A0 to A1. Exactly backwards. The fix is one line: take the minimum, and treat missing evidence as zero. Missing evidence is now the worst case, not a free pass.

Benchmarks lie. Our first verification runs were non-deterministic — coefficient of variation 0.297 on one run, 0.193 on the next, flipping the verdict. We had to separate "this got worse" from "we couldn't tell," and make noise resolve to inconclusive rather than silently passing or failing. A verifier that's confidently wrong is worse than no verifier.

Gemini's API moved under us. The Interactions API rejected our pinned SDK outright — "legacy Interactions API schema is no longer supported." And google-genai 2.x garbage-collects a Client the moment nothing references it, so every inline call died mid-request with "Cannot send a request, as the client has been closed." Every test passed the whole time, because every test mocked the client. Nothing catches that except calling the real API.

Merge conflicts at 2am with four people and one shared contracts package. Frozen Pydantic models saved us more than once — you cannot quietly change a contract another engineer depends on.

Accomplishments that we're proud of

537 tests passing. Not a demo held together with a DEMO_MODE flag.

The measurement is real. No mocked numbers anywhere in the verification path. Docker containers, k6 load, actual SQL query counts: 2 → 51 → 2.

Lou degrades honestly. When Gemini returned a 500 after six minutes, the run came back abandoned with the reason recorded — not a fake answer, not a crash. The system's failure modes are as designed as its success path.

We found our own safety bug and fixed it instead of hoping the demo wouldn't hit it.

What we learned

Mocks validate your assumptions, not the API. Three real Gemini bugs shipped straight through a fully green test suite. We've started asserting on error shape — status codes, builtin exception types — rather than SDK class names, so the next reshuffle can't break us the same way.

"Confidence" needs a direction. Any aggregate that can be improved by removing information is the wrong aggregate. That's a general lesson about scoring systems, not a Python bug.

Two risks, not one. The cost of leaving a problem and the danger of changing it are different numbers. Collapsing them into "risk" makes the system either reckless or useless.

Protocol boundaries are how four people ship in 36 hours.

What's next for Lou

Cut Gemini latency. Six minutes on a 32KB evidence bundle isn't demoable — the context budget knobs exist, we need to tune them.

Wire the orchestrator to the CLI. The remediation orchestrator runs the full detect→fix→verify loop today but is only reachable from our integration gate.

Climb to A3. Open real PRs with the evidence bundle attached, so a human reviews a claim with proof instead of a diff.

Continuous mode. Watch main, build a debt ledger over time, show which modules are compounding.

Learn from outcomes. Every merged or rejected fix is a labeled example. Feed accept/reject rates back into the confidence model, and Lou earns autonomy instead of being granted it.