FollowThrough - Autonomous Meeting Agent

Meetings decide who does what. FollowThrough makes sure it happens: files the issues, watches for blockers, escalates to a human — and never sends email without approval.

Comment

Inspiration

Every team has the same bug, and it isn't in the code.

The meeting ends. Everyone agreed on who does what. Then Slack moves on, the tab closes, and a week later somebody asks "wait, did anyone actually do the auth API?" Nobody tracked it, because tracking it is nobody's job.

There are plenty of tools that summarise a meeting. A summary tells you what was said. The expensive question is what happens over the next week — and whether anyone notices when it doesn't.

We wanted an agent that owns the week after the meeting.

What it does

FollowThrough takes a raw meeting transcript and runs the follow-through by itself:

  1. Extracts commitments — task, owner, deadline, and the dependencies between them. "I'll review it once it's ready" becomes a machine-readable edge, even though nobody said the word "dependency".
  2. Files the work — creates real GitHub issues, assigns them, and comments on each one explaining how it's being tracked.
  3. Respects the order — a task whose dependency isn't done parks as WAITING instead of nagging someone about work they can't start.
  4. Watches — polls on its own 5-second clock for blocked labels, missed deadlines, and merged pull requests. Nobody has to tell it anything.
  5. Escalates — when something is stuck, Murph (our voice/chat agent) drafts an email to the person who can unblock it.
  6. Verifies — a commitment is done when its linked PR merges, not when somebody claims it is. When that happens, whatever was waiting on it starts automatically.

In our demo run, the system logged 37 pipeline events. A human performed two of them: uploading the transcript, and clicking Approve. GitHub issue #20 was filed by the system for a task nobody asked it to start.

The one thing we'd want you to look at

The agent cannot send email. Not "is told not to" — cannot.

Our first version had the classic mistake. The approval gate was one sentence of prompt text:

"do NOT call send_approved_escalation unless the user has explicitly said yes in this conversation"

And send_approved_escalation was an ordinary tool on the agent. Nothing but the model's cooperation stood between it and sending mail on a team's behalf. A prompt injection in a transcript, a hallucinated "the user said yes", or a model update that reads the instruction differently, and it fires.

So we moved the gate out of the prompt and into state:

Murph drafts  ──▶  PENDING_APPROVAL  ──approve──▶  APPROVED  ──▶  SENT
                         │                             ▲
                         └──reject──▶ REJECTED         │
                                                       │
                            only POST /escalations/{id}/approve
                            writes this state, and no agent has
                            a tool that reaches it

send_escalation is a tool Murph can call. It refuses anything not already APPROVED. Our test suite asserts the property directly rather than trusting it:

PASS: draft is queued as PENDING_APPROVAL, not sent
PASS: agent cannot self-approve: still PENDING_APPROVAL after 'yes send it'
PASS: human approval sends it, commitment state updated

In the demo we tell Murph "I approve it, send the email now" and it replies:

"The draft is still waiting for your approval in the dashboard. I cannot send it until you click Approve there."

That's not the model being polite. It's the model having no route to the approved state.

How we built it

Four agents, decoupled over Pub/Sub rather than direct calls:

Transcript
    │
    ▼
Meeting Intelligence (Gemini)  ──▶  Firestore
    │
    ▼  Pub/Sub: meeting-processed
    │
Execution Agent (ADK LlmAgent + GitHub tool)  ──▶  GitHub Issues
    │
    ▼
Monitoring Agent (own clock)  ──▶  blockers, deadlines, merged PRs
    │
    ▼
Murph (ADK LlmAgent, voice + chat)  ──▶  drafts escalations
    │
    ▼
Approval gate (human only)  ──▶  the email sends, or it doesn't
  • Gemini for extraction and for both ADK agents — three separate models so each has its own quota headroom.
  • Google ADK for the tool-calling agents (Execution, Murph).
  • Firestore for commitment, event, and escalation state.
  • Pub/Sub for agent-to-agent decoupling.
  • FastAPI backend, React + Vite control-room dashboard.
  • Web Speech API for talking to Murph.
  • Cloud Run deployment with Secret Manager.

Every external dependency sits behind an in-memory twin with an identical function signature, selected per service by an environment flag. That let us build and test the entire architecture with zero credentials, and it's why the whole thing is provable by one command:

python smoke_test.py     →     ALL PASSED (8/8)

Challenges we ran into

The demo would have frozen at its climax. Our monitoring agent polled GitHub only for commitments in IN_PROGRESS. So the moment something became BLOCKED — and therefore ESCALATED — GitHub was never checked for it again. A merged PR was never noticed, a removed blocked label was never seen, and every downstream task waited forever. Our state machine documented ESCALATED → IN_PROGRESS; no code implemented that edge. It's the exact scenario the product exists for: you escalate, a human unblocks you, work resumes. We only found it by running the UI — it was invisible in review.

The deploy script could not have worked. Three required environment variables were never passed to Cloud Run. It referenced Secret Manager secrets nothing created. And it used --min-instances 0 with default CPU throttling — which would have deployed a service that returned 200 on /health, served the dashboard, and quietly performed no monitoring at all, because the monitoring loop and Pub/Sub subscriber are background threads that Cloud Run suspends between requests. It would have looked fine right up until the demo.

"Kartikeya@example.com". Asked to escalate to Kartikeya, the model confidently produced an email address that doesn't exist — the transcript never contained one. Approving that draft would have sent mail into the void while telling a human it was delivered. Recipients are now resolved from config, and model-supplied addresses on placeholder domains are rejected.

All-or-nothing offline mode. Our original flag meant you needed a GCP project with billing linked before you could create one real GitHub issue. We split it into five per-service flags, so Firestore and Pub/Sub can stay in-memory while GitHub, Gemini and Gmail are genuinely live.

What we learned

Static review doesn't find the bugs that break demos. Both of our worst defects — the dead-end state machine and the silently-disabled monitoring — looked completely reasonable on the page. Running it found them in minutes.

Safety claims must be enforced in state, not in prompts. A prompt is a request. If your safety property matters, make the unsafe path unrepresentable and write a test that proves it.

Autonomy is more credible with a boundary than without one. Deleting our approval gate would make the agent "more autonomous" and considerably worse. The interesting design question isn't how much an agent can do alone — it's where it should stop.

What's next

  • Multi-level dependency graphs rather than single-level chains
  • Subscribers on the task-created / task-blocked / task-verified topics — they're published today as the extension seam for a Slack notifier
  • Cloud Scheduler → Eventarc instead of the in-process polling loop
  • Calendar integration, so deadlines resolve against real working days

Honest about the limits

Judges find these anyway, so: Firestore and Pub/Sub run in-memory in the recorded demo — the full deployment path exists in deploy.sh and firestore.indexes.json, but we scoped the video to keep setup to minutes. GitHub, Gemini and Gmail are making real API calls on camera, and the dashboard's LIVE chip reports exactly which services are real at any moment. Monitoring polls rather than using Cloud Scheduler. There is no API authentication; we restricted CORS instead, which was the part that actually mattered once an approval endpoint existed.

Built With

Share this project:

Updates