Inspiration
We vibe-coded for an afternoon. Came back to 47 commits, three sub-agent dispatches we didn't remember kicking off, and a feeling of "did this actually do what I asked?" — but no good way to find out. Git showed us what changed. Nothing showed us why.
Every coder we've talked to has the same after-vibe-coding ritual: scroll the diff, squint at it, pretend you understand, merge anyway. I wanted the opposite — to actually read the session back, follow the agent's thinking, and only trust what we could verify.
What it does
VibeCheck takes a Claude Code session transcript (the raw JSONL) plus the resulting git history and turns it into a navigable, queryable replay of the agent's thinking.
- Flow — a force-directed graph of every sub-agent dispatch, showing which agent kicked off which child, what they found, and which result got picked. This is the landing tab — the shape of the agent's work is the first thing you see.
- Story — every decision the agent made, in order, as expandable cards.
- Code — diff viewer with hover-to-reveal reasoning overlays on individual lines.
- Patterns — recurring agent behaviors the parser auto-detected (ralph loops, critic→fixer cycles, retry storms).
- Ask — chat the run: "why did you pick X over Y?" answered against the full transcript.
- Rejected alternatives — the approaches the agent tried for 20 minutes and abandoned, surfaced inline so you see what got thrown away, not just what survived.
Tagline: Vibe coding is fast. Understanding what got built is slow. VibeCheck makes the second part fast too.
How we built it
End-to-end flow: session.jsonl + repo URL → Parser (Python, streams the file, redacts secrets, detects patterns, builds a DecisionNode tree, correlates git commits by timestamp) → Redis Cloud cache (with in-memory LRU fallback) → FastAPI service exposing REST + GraphQL + SSE → React SPA rendering Flow, Story, Code, Patterns, and Ask tabs. The whole stack ships as a single Chainguard zero-CVE container, deployed to Akash Console.
The stack:
- Parser (Python) — streams JSONL line-by-line so 100k+ event sessions don't blow memory; redacts API keys /
.envpaths before anything is cached; correlates git commits to decision nodes via timestamp windows. - API (FastAPI) —
/session/ingest,/session/:id,/session/:id/node/:id,/session/:id/ask,/session/:id/stream, plus a healthz/stats panel. - Cache (Redis Cloud) — parsed trees can be 5–50 MB; Redis keeps hot sessions snappy with a 24h TTL and silent in-memory fallback when Redis is unreachable.
- GraphQL (Strawberry → WunderGraph Cosmo subgraph) — federation-ready, lets you ask structured questions like "only sub-agent dispatches in round 7."
- Inference (AkashML) — Llama-3.1-8B-FP8 powers Ask, OpenAI-compatible adapter pattern, Anthropic as fallback.
- Frontend (React + Vite + Tailwind + D3) — five tabs (Flow, Story, Code, Patterns, Ask), keyboard-first navigation, Flow is the landing view.
- Container (Chainguard zero-CVE base) — multi-stage Dockerfile: Chainguard
node:latest-devbuilds the SPA, Chainguardpython:latest-devinstalls deps, Chainguardpython:latest(no shell in the runtime image) runs uvicorn. - Hosting (Akash Console) — one SDL file, deployed to a decentralized provider, public URL in ~60 seconds.
Challenges we ran into
- Claude Code JSONL is a firehose. Tool calls, sub-agent dispatches, reasoning blocks, and assistant text are all interleaved with no clean nesting. Building a sane decision tree out of it took longer than parsing the bytes ever would.
- Linking commits to decisions. Git has no idea which agent turn produced which commit. We correlate via timestamp proximity, which is right ~95% of the time and obviously wrong the other 5% — and visualizing that uncertainty without misleading the user was its own UX problem.
- Secret leakage in transcripts. Real sessions contain
.envpaths, API keys, internal URLs, even source snippets with hardcoded tokens. We had to write a redactor before we dared upload our own session for the demo. - Rejected alternatives are invisible to git. The agent's "I considered approach A and rejected it after 20 minutes" only exists in the transcript, never in the diff. Surfacing this without making the UI cluttered was a design problem, not a parsing problem.
- Sponsor wiring under time pressure. Cosmo subgraph publish, Akash SDL syntax, Chainguard multi-stage layering — each had a quirk that ate 15–30 minutes. "Checkbox sponsor integration" and "actually working sponsor integration" are very different time budgets.
- Default tab matters more than we expected. We shipped with Story as the landing tab and watched users bounce — they didn't know what they were looking at. Switching the default to Flow changed the demo entirely: the shape of the agent's work is legible in 2 seconds.
Accomplishments that we're proud of
- The sponsor stack is actually wired, not boxes ticked. Redis genuinely caches the hot path. Cosmo is a real subgraph, not a wrapper. Chainguard is in the runtime image. AkashML answers the chat. Akash hosts the whole thing.
- Opening a 50 MB parsed session is sub-second because of the Redis layer.
- The "rejected alternatives" surface shows something no other tool shows — work the agent did and threw away.
- "Ask the run" gives coherent, grounded answers, not vibes — because it's reading the full structured tree, not a flat transcript.
- The container is one Dockerfile on a hardened, zero-CVE base image, deployed on a decentralized cloud, with a public URL.
- We built a parser robust enough to handle our own ralph-loop sessions — sub-agents inside sub-agents, malformed lines, mid-stream tool errors — without crashing.
What we learned
- Agents leave way more reasoning behind than people realize — it's just trapped in JSONL no one reads.
- A good replay tool is mostly information design, not ML. The hard part is deciding what to show, not what to compute.
- Sponsor stacks compose surprisingly well when you pick them based on what your app actually needs (cache, inference, GraphQL, container, host) instead of trying to wedge them in.
- Vibe coding produces a qualitatively different code review problem than human PRs. Reviewing volume isn't the issue — reviewing intent is.
- Defaults are everything in a 90-second demo. The first tab a user lands on either makes the product legible or invisible.
What's next for VibeCheck
- Live mode — stream sessions in real time as the agent runs, so you can intervene mid-loop instead of post-mortem.
- Direct-write SDK — a tiny
vibecheck.decision(...)library so devs can emit decisions and rejected alternatives from their own code or custom agent loops, without needing a Claude Code JSONL at all. - More agent frameworks — Cursor agents, Aider, custom ralph loops, OpenAI Assistants.
- Cross-session diffs — "compare what the agent did this run vs. last run on the same prompt."
- Regression detection — flag when an agent silently abandoned a pattern it used to follow.
- Team sharing — drop a VibeCheck link into a PR review so your reviewer sees the thinking, not just the diff.
- IDE plugin — see decision context inline while reading the code.
Built With
- akash
- akashml
- chaingaurd
- python
- redis
- redis-cloud
- typescript
- wundergraph
Log in or sign up for Devpost to join the conversation.