Debrief

Debrief is a browser-native AI chief of staff. It eliminates the gap between finding information and acting on it — without breaking your flow.

Comment

Inspiration

The average power user has 10–30 browser tabs open at any given time — not out of disorganization, but because the browser has become a makeshift memory system. You find something, you don't have time to act on it, so you leave the tab open. Then another. Then another. The tab bar becomes a to-do list you never fully clear.

We built Debrief because the gap between finding information and doing something with it is where productivity dies. Every existing tool addresses one side: note apps capture, automation tools route, AI assistants answer questions. None of them close the loop inside the browser, in the moment, without breaking your focus. We wanted to build the thing that does all three — and does it the moment you speak.

What it does

Debrief is a browser-native AI chief of staff. It turns the browser into an active operational layer through three core workflows:

Surgical Capture — Highlight any text on any page, press a hotkey, and speak your intent. A lightweight modal confirms the capture without disrupting your layout. Your words and the selected content are saved instantly to a persistent ledger.

The Staged Ledger — A sidebar that holds every captured item with its source, content, and spoken intent. It separates the act of finding from the act of routing — you stay in flow until you're ready to act, and nothing gets lost.

Autonomous Execution — When you speak an intent, Debrief doesn't wait. It runs web searches, sends Amazon Nova Act to navigate live pages autonomously, enriches captures with research, and routes items directly to Notion, Todoist and more — without you leaving the page you're on.

One command can chain multiple outputs: capture a product, find similar alternatives, compare pricing — all staged as separate ledger entries from a single voice instruction.

How we built it

Debrief is a monorepo built across four layers:

  • Chrome Extension (MV3) — handles hotkey capture, the Shadow-DOM modal, the sidebar ledger, and a full-duplex WebSocket audio pipeline to the backend. Audio is streamed in binary frames; control signals travel as JSON text frames over the same connection.

  • FastAPI Backend — manages auth, sessions, billing, and the Bedrock streaming connection to Amazon Nova Sonic. LangGraph is imported directly into the same process — no HTTP boundary, no added latency.

  • LangGraph Orchestration — a standalone Python package (debrief-agent) that handles pre-session memory injection, intent routing, tool dispatch (web search, Nova Act), and InsightPayload schema enforcement via Pydantic.

  • Next.js Dashboard — session history, ledger views, and settings accessible outside the extension.

Nova Sonic handles voice — it receives the audio stream, interprets intent, and speaks back through the same WebSocket. Nova Act handles browser-level execution — autonomous navigation, form interaction, and page scraping on live sites.

Challenges we ran into

Full-duplex audio over WebSocket — the browser's WebSocket API doesn't distinguish well between binary and text frames at the application layer. Routing audio chunks and control signals over a single connection without collision required careful frame-typing and ordering logic on both ends.

LangGraph in-process with FastAPI — removing the HTTP boundary between orchestration and the API server simplified the architecture but introduced shared state risks. We had to be deliberate about session isolation so concurrent users didn't bleed into each other's graph runs.

Nova Act on real pages — autonomous agents behave differently on production websites than in controlled environments. Dynamic content, cookie banners, and lazy-loaded elements all required defensive handling and fallbacks to avoid silent failures mid-task.

Accomplishments that we're proud of

  • A working full-duplex voice pipeline where Nova Sonic both receives and speaks back — including barge-in support, where the user can interrupt mid-response and the backend halts immediately.
  • Chained agentic execution from a single voice command — one intent producing multiple enriched ledger entries in parallel.
  • A clean four-layer architecture where each component has a single responsibility and the boundaries are explicit.
  • The InsightPayload schema enforced at every layer — LangGraph Pydantic, PostgreSQL JSONB, FastAPI schema, and TypeScript interface — so structured AI output never degrades silently.

What we learned

Voice-first interfaces expose latency problems that text interfaces hide. A 300ms delay feels acceptable in a chat UI; in a voice conversation it feels broken. This forced us to optimize the WebSocket pipeline earlier than we expected.

Agentic systems require failure modes that are visible to the user. When Nova Act can't complete a task, the ledger item needs to reflect that honestly — not silently drop or pretend it succeeded. Building trust into the UI is as important as building capability into the backend.

Structured output schemas are worth the overhead. Enforcing InsightPayload at every layer felt like over-engineering at first. In practice it caught schema drift between the LangGraph node and the frontend three times before it would have surfaced as a runtime bug.

What's next for Debrief

  • Scheduled resolution — ledger items that trigger on a schedule, not just on command. "Every Monday, compile everything tagged 'newsletter' and draft it in Notion."
  • Price and content monitoring — Nova Act revisiting a saved URL on a recurring basis and alerting when something changes.
  • Team ledgers — shared capture spaces for collaborative research workflows, with attribution and routing rules per member.
  • Mobile companion — extending the capture layer beyond the desktop browser to phone screenshots and voice memos, all feeding the same ledger.
  • Expanded routing targets — Linear, Google Calendar, email drafting, and Slack thread replies as first-class resolution destinations.

Built With

Share this project:

Updates