GitShell

GitShell transforms the world’s unresolved developer questions into trusted, living documentation.

Comment

Inspiration

Popular open-source repos drown in the same recurring questions because the docs never catch up. Maintainers burn out triaging duplicates, users can't find authoritative answers, and AI assistants hallucinate because nothing citeable exists. We wanted an autonomous agent that closes the loop — issue signal → clustered gap → paid, published, citeable doc — and earns its keep along the way.

What it does

GitShell points at any GitHub repo, scrapes recent issues, clusters them into recurring documentation gaps (with severity and confidence), then opens a paid agent workflow: a maintainer (or another agent) pays per gap, GitShell drafts a fix, a human approves it, and it gets published as a citeable knowledge artifact linked back to the originating issues.

How we built it

GitShell is a FastAPI agent orchestrated with LangChain + LangGraph, with every sponsor wired to a specific job:

  • Datadog — APM traces plus LLM Observability spans for every tool call, model invocation, and payment event, so the agent is fully introspectable end-to-end.
  • Senso — human-in-the-loop publishing of citeable answers into a structured knowledge layer that LLMs can cite.
  • Nimble — web scraping fallback for official docs when GitHub signal alone isn't enough.
  • ClickHouse — durable audit trail of gaps, runs, payments, and analytics.
  • Stripe + x402 / MPP — agent-native payment rails. The /runs endpoint is gated behind HTTP 402 Payment Required; severity-priced Stripe Checkout sessions (low / medium / high) unlock the publish step, and receipts flow back into run metadata.
  • Supporting stack: GitHub issues API, OpenAI + Gemini for clustering and drafting, HTMX for the live event stream UI.

Agent payments

Payments aren't bolted on they're the spine of the workflow:

  • Every run hits a require_payment dependency that returns HTTP 402 unless a valid receipt is presented, matching the x402 / Merchant Payment Protocol pattern emerging for agent-to-agent commerce.
  • Each documentation gap is priced by severity via price_for_severity, and a Stripe Checkout session is created per gap with run_id and gap_index baked into metadata.
  • The publish step verifies the session server-side before Senso ever sees a draft — so the agent only spends compute and publishes citeable artifacts that have actually been paid for.
  • A dev bypass header (X-Dev-Payment: paid) keeps local iteration fast without weakening the production gate.

The result: an autonomous agent that can be hired, paid, and held accountable per unit of work.

Challenges we ran into

Squeezing a real autonomous loop into one day meant ruthless scope cuts. Getting Datadog LLM Observability to capture every model call cleanly through ddtrace-run took iteration. Threading Stripe metadata through an async LangGraph run and back into the publish step — without leaking unpaid drafts to Senso — was the trickiest piece. And designing a publish review that felt like real human-in-the-loop, not a rubber stamp, was harder than the clustering itself.

Accomplishments that we're proud of

  • A working end-to-end demo in under 60 seconds on a real repo.
  • A live 402 → Stripe Checkout → publish flow that demonstrates agent payment rails, not a mock.
  • Lit-up sponsor pills in the UI showing every tool call in real time.
  • Full LLM observability in Datadog for every span the agent emits.
  • A Senso publish flow that produces a genuine citeable answer, linked back to the source GitHub issues.

What we learned

Autonomy without observability is unshippable — Datadog LLM Obs turned the agent from a black box into something we could debug. Payments belong inside the agent loop, not around it: pricing per gap forced us to be honest about what an "agent output" actually is. And the right unit of agent output is a citeable artifact, not a chat message — Senso made that concrete.

What's next for GitShell

  • Auto-comment back on the originating GitHub issues with the published Senso answer.
  • Expand Nimble scraping to ingest official docs as priors for clustering.
  • Ship a Datadog dashboard template for agent runs, payments, and gap analytics.
  • Move from Stripe Checkout to fully programmatic x402 settlement for true agent-to-agent payments.
  • Open multi-repo monitoring so maintainers can subscribe their projects on a recurring plan.

Built With

Share this project:

Updates

Submission history