Inspiration

We vibe-coded for an afternoon. Came back to 47 commits, three sub-agent dispatches we didn't remember kicking off, and a feeling of "did this actually do what I asked?" — but no good way to find out. Git showed us what changed. Nothing showed us why.

Every coder we've talked to has the same after-vibe-coding ritual: scroll the diff, squint at it, pretend you understand, merge anyway. I wanted the opposite — to actually read the session back, follow the agent's thinking, and only trust what we could verify.

What it does

VibeCheck takes a Claude Code session transcript (the raw JSONL) plus the resulting git history and turns it into a navigable, queryable replay of the agent's thinking.

  • Flow — a force-directed graph of every sub-agent dispatch, showing which agent kicked off which child, what they found, and which result got picked. This is the landing tab — the shape of the agent's work is the first thing you see.
  • Story — every decision the agent made, in order, as expandable cards.
  • Code — diff viewer with hover-to-reveal reasoning overlays on individual lines.
  • Patterns — recurring agent behaviors the parser auto-detected (ralph loops, critic→fixer cycles, retry storms).
  • Ask — chat the run: "why did you pick X over Y?" answered against the full transcript.
  • Rejected alternatives — the approaches the agent tried for 20 minutes and abandoned, surfaced inline so you see what got thrown away, not just what survived.

Tagline: Vibe coding is fast. Understanding what got built is slow. VibeCheck makes the second part fast too.

How we built it

End-to-end flow: session.jsonl + repo URL → Parser (Python, streams the file, redacts secrets, detects patterns, builds a DecisionNode tree, correlates git commits by timestamp) → Redis Cloud cache (with in-memory LRU fallback) → FastAPI service exposing REST + GraphQL + SSE → React SPA rendering Flow, Story, Code, Patterns, and Ask tabs. The whole stack ships as a single Chainguard zero-CVE container, deployed to Akash Console.

The stack:

  • Parser (Python) — streams JSONL line-by-line so 100k+ event sessions don't blow memory; redacts API keys / .env paths before anything is cached; correlates git commits to decision nodes via timestamp windows.
  • API (FastAPI) — /session/ingest, /session/:id, /session/:id/node/:id, /session/:id/ask, /session/:id/stream, plus a healthz/stats panel.
  • Cache (Redis Cloud) — parsed trees can be 5–50 MB; Redis keeps hot sessions snappy with a 24h TTL and silent in-memory fallback when Redis is unreachable.
  • GraphQL (Strawberry → WunderGraph Cosmo subgraph) — federation-ready, lets you ask structured questions like "only sub-agent dispatches in round 7."
  • Inference (AkashML) — Llama-3.1-8B-FP8 powers Ask, OpenAI-compatible adapter pattern, Anthropic as fallback.
  • Frontend (React + Vite + Tailwind + D3) — five tabs (Flow, Story, Code, Patterns, Ask), keyboard-first navigation, Flow is the landing view.
  • Container (Chainguard zero-CVE base) — multi-stage Dockerfile: Chainguard node:latest-dev builds the SPA, Chainguard python:latest-dev installs deps, Chainguard python:latest (no shell in the runtime image) runs uvicorn.
  • Hosting (Akash Console) — one SDL file, deployed to a decentralized provider, public URL in ~60 seconds.

Challenges we ran into

  • Claude Code JSONL is a firehose. Tool calls, sub-agent dispatches, reasoning blocks, and assistant text are all interleaved with no clean nesting. Building a sane decision tree out of it took longer than parsing the bytes ever would.
  • Linking commits to decisions. Git has no idea which agent turn produced which commit. We correlate via timestamp proximity, which is right ~95% of the time and obviously wrong the other 5% — and visualizing that uncertainty without misleading the user was its own UX problem.
  • Secret leakage in transcripts. Real sessions contain .env paths, API keys, internal URLs, even source snippets with hardcoded tokens. We had to write a redactor before we dared upload our own session for the demo.
  • Rejected alternatives are invisible to git. The agent's "I considered approach A and rejected it after 20 minutes" only exists in the transcript, never in the diff. Surfacing this without making the UI cluttered was a design problem, not a parsing problem.
  • Sponsor wiring under time pressure. Cosmo subgraph publish, Akash SDL syntax, Chainguard multi-stage layering — each had a quirk that ate 15–30 minutes. "Checkbox sponsor integration" and "actually working sponsor integration" are very different time budgets.
  • Default tab matters more than we expected. We shipped with Story as the landing tab and watched users bounce — they didn't know what they were looking at. Switching the default to Flow changed the demo entirely: the shape of the agent's work is legible in 2 seconds.

Accomplishments that we're proud of

  • The sponsor stack is actually wired, not boxes ticked. Redis genuinely caches the hot path. Cosmo is a real subgraph, not a wrapper. Chainguard is in the runtime image. AkashML answers the chat. Akash hosts the whole thing.
  • Opening a 50 MB parsed session is sub-second because of the Redis layer.
  • The "rejected alternatives" surface shows something no other tool shows — work the agent did and threw away.
  • "Ask the run" gives coherent, grounded answers, not vibes — because it's reading the full structured tree, not a flat transcript.
  • The container is one Dockerfile on a hardened, zero-CVE base image, deployed on a decentralized cloud, with a public URL.
  • We built a parser robust enough to handle our own ralph-loop sessions — sub-agents inside sub-agents, malformed lines, mid-stream tool errors — without crashing.

What we learned

  • Agents leave way more reasoning behind than people realize — it's just trapped in JSONL no one reads.
  • A good replay tool is mostly information design, not ML. The hard part is deciding what to show, not what to compute.
  • Sponsor stacks compose surprisingly well when you pick them based on what your app actually needs (cache, inference, GraphQL, container, host) instead of trying to wedge them in.
  • Vibe coding produces a qualitatively different code review problem than human PRs. Reviewing volume isn't the issue — reviewing intent is.
  • Defaults are everything in a 90-second demo. The first tab a user lands on either makes the product legible or invisible.

What's next for VibeCheck

  • Live mode — stream sessions in real time as the agent runs, so you can intervene mid-loop instead of post-mortem.
  • Direct-write SDK — a tiny vibecheck.decision(...) library so devs can emit decisions and rejected alternatives from their own code or custom agent loops, without needing a Claude Code JSONL at all.
  • More agent frameworks — Cursor agents, Aider, custom ralph loops, OpenAI Assistants.
  • Cross-session diffs — "compare what the agent did this run vs. last run on the same prompt."
  • Regression detection — flag when an agent silently abandoned a pattern it used to follow.
  • Team sharing — drop a VibeCheck link into a PR review so your reviewer sees the thinking, not just the diff.
  • IDE plugin — see decision context inline while reading the code.

Built With

Share this project:

Updates

Submission history