agentic outbound agency

Five ADK agents and an LLM judge research companies, write cold emails, and handle replies. None can send, approve, or overwrite without passing through a single audited gate, sophisticated harness.

Comment

Project name

Outbound Agency — an outbound agent with real autonomy and no unsafe moves.

Elevator pitch (one line)

Five ADK agents and an LLM judge research companies, write cold emails, and handle replies — and none of them can send, approve, or overwrite anything without passing through a single audited gate.

(Judging weights, for reference: Innovation & Operational Utility 40%, Architectural Discipline & Tech Stack 30%, Demo & Production Readiness 30%. This writeup is ordered to hit all three, roughly in that priority.)

Track

The Taskmaster. The track's own stated focus: "an event-driven workflow with autonomous routing... watching for a change, figuring out what needs to happen next, and interacting with different apps to get the job done, from start to finish, without you guiding each step." Its own example use case — "an agent monitoring inbox inquiries, checking calendar availability, drafting proposals from previous work samples, and saving them for human review" — is this project, mirrored: outbound instead of inbound, but the same shape (watch → judge → draft → hold for a human).

Inspiration

As a solo business builder, my pain every day is contacting related businesses and stakeholders for a reach-out, a comment, or a demo booking. I used to do this manually — researching and emailing 40-50+ companies, and manually reviewing every single reply. I know there are AI outreach tools out there, but most are expensive, hard to control, unaware of what my actual product does, and can't be customized — with a high failure and reject rate from email warm-up and provider policy. That's what inspired this project. Cold outreach is the obvious agent use case and the one most likely to go wrong. An agent that can research a company and write a convincing email is also an agent that can email the wrong person, email them twice, or email someone who already asked it to stop. We wanted to find out whether you can give agents genuine judgement — real tool-choosing, real verdicts, no scripted pipeline — and still be able to prove, structurally, that they cannot do the damaging thing.

What it does

One sentence in, four stages out:

python -m app.taskmaster_cli --task "run outreach for the HK therapy clinics offer, 10 targets"

A root ADK LlmAgent plans and dispatches: a ResearchAgent that chooses its own tools (google_search, url_context, page fetch) until it can fill a company profile; a signal detector that must quote its evidence; an ICPJudgeAgent that issues the final fit verdict and must justify any divergence from the deterministic score; and a DraftAgent — an ADK LoopAgent where a writer and a critic iterate up to three times, every version stored.

Then it stops, and reports that N targets are awaiting human review. It cannot go further.

After a human approves in the console, a 19-item preflight runs, the email is written to a DRY_RUN outbox, replies are classified by a dedicated classifier agent, and a deterministic router suppresses, escalates, or queues a follow-up.

The natural-language interface is state-aware, not a scripted wrapper. The Taskmaster root agent has six tools, not one: report_pipeline_status (read-only — target counts by state, what's awaiting review, the kill switch's live state, recent refusals) plus five per-stage tools (import_and_research, draft_for_scored, dry_run_send_approved, fetch_and_classify_replies, and resume_pending_research). When a run is interrupted — a timeout, a dropped connection, anything — the next invocation checks real database state first and picks up at the correct stage, never re-importing or re-scoring a target that already finished. We hit this for real during the build twice: a 10-company batch exceeded its wall-clock ceiling mid-run, and a follow-up "continue" correctly resumed the surviving state — but a second re-run that explicitly re-imported the same CSV collided with Postgres's accounts.normalized_domain UNIQUE constraint and refused the whole batch. resume_pending_research (added live, mid-build, once we found the gap) is the actual fix: it drives already-imported targets straight through the research stage by target_id, never touching import_csv, so there is nothing left to collide with. The model diagnosed the root cause correctly on its own before this tool existed — it just had no way to act on the diagnosis. Now it does.

This is what makes the parts of the system that are SUPPOSED to run unattended actually run unattended: the agent self-retries and checks its own "am I actually finished" against real state, specifically for the two ways this used to go wrong — a timeout firing mid-batch, or the agent assuming everything's done when it isn't. It's the same shape as Claude Code's own /goal command, but patched with a hard max-iteration cap so it can't run forever. For the parts of the job that genuinely don't need a human — research, scoring, drafting — an operator can give it one task and walk away, and only has to come back to a "finished" or "awaiting_review" notification, never babysit the run or hit retry by hand. Everything past that point (the send, the approval) still stops for a human on purpose — this is autonomy up to the review gate, never autonomy through it. That distinction is something this hackathon taught me to be explicit about in any agentic system, not just this one.

Crash continuity is tested, not just claimed. python -m app.autonomous_taskmaster wraps the Taskmaster in a bounded loop: call it, then decide whether to call it again by reading the same selector functions the stage tools already select through — never by trusting the model's own "am I done." --max-iterations (default 30) keeps it honestly bounded instead of running forever. Three tests prove the actual behavior, not just the intent: nothing pending skips the loop entirely (no wasted model call); a target seeded at new is driven, through the real state machine, across exactly as many simulated crash-and-resume cycles as it needs before the loop reports done; and a target that a stubbed invocation never advances hits the iteration bound and fails loudly, on stderr, rather than hanging. Underneath the loop, every stage tool already isolates failure per target — one crashing company is logged, transitioned to failed, and never aborts the rest of the batch (_record_target_crash, exercised across five separate dispatch points: research, drafting, sending, reply classification, and recovery itself).

Every noteworthy capability:

  • A public, zero-credential surface (/rules, /demo, /test-run) — pre-rendered to disk ahead of time, so these pages open no database connection at request time and are safe to leave reachable indefinitely. Everything else (the full target list, the review queue, the kill switch) stays behind the console's API key. The carve-out is an enumerated set of exact literal paths in the same global auth dependency that protects everything else — never a wildcard.
  • Multi-vertical, proven, not just claimed. A second live offer (ai-security — real AI agent companies: LangChain, CrewAI, LlamaIndex, and others) was added and run end-to-end on the exact same code as the therapy vertical: one new YAML file, zero new machinery.
  • Adversarial-tested. A fixed 12-attack corpus (prompt injection, forged authority, data exfiltration, footer stripping, no-approval send, suppression evasion via case/plus-tag variants, replay, kill-switch abuse, a Taskmaster over-claiming approval) drives the real pipeline with only the model factories stubbed. All 12 pass.
  • 9-way reply routing, confidence-gated. A reply classifies into one of nine outcomes (positive, not_now, negative, unsubscribe, wrong_person, objection, meeting_request, risky, unclear); a classification below the confidence floor, or the risky class itself, is routed to mandatory human review — it can never auto-act, no matter how the router is called.
  • No send transport anywhere, enforced not asserted. A test walks every module in the repo and fails the suite if an SMTP or mail-sending import ever appears. There is no live-send code path to accidentally enable, and no config flag that would turn one on.
  • One connection seam, two real dialects. app/db.py speaks SQLite (local, tests) and Postgres/Cloud SQL (production) behind the same connect() call — no dialect-specific code anywhere else in the pipeline.

The twist: autonomy above, determinism below. The design rule is one line: LLM agents only ever produce verdicts; deterministic code performs every action.

The draft loop makes it concrete. The writer and critic are both LlmAgents. The third member of the loop is plain Python — it owns the gated write, both state transitions, and the loop-exit signal. The critic emits a judgement; code decides what that judgement does. The email schema has no footer field, so a model cannot omit a compliance footer — code composes it.

That rule is enforced by construction, not convention:

  • One write path. Every core-table write goes through write_gate.commit(action, agent_id), which refuses unknown actions and unauthorised principals before any SQL runs. The Taskmaster's registry entry has an empty capability set — no gated write may ever be attributed to it.
  • One state path. Every state change goes through state_machine.transition(), validated against an explicit transition table.
  • A kill switch that fails closed. It's a file, read uncached. A missing or malformed file counts as ENGAGED — deleting it halts everything rather than disabling the halt. Engaged, it refuses approvals but still allows reject and suppress: the brakes must work after the emergency stop.
  • A read-only console. Two AST tests parse the console's own source: one refuses any raw write-SQL string anywhere in it; the other checks its imports against an ALLOWLIST of exactly the two modules whose write functions the console is permitted to call (the review gate, the kill-switch toggle) — any other write-path import anywhere in the console fails the test immediately. Its only doors are the five review decisions and the kill-switch toggle.

Every guarantee above is checked by a test that tries to break it, not just asserted in a comment. The live console has a /rules page — the scoring formula, all nine policy rules, and the full state-transition table, in one screen, hand-verified against the real source rather than imported (importing would have meant widening the console's own audited zero-write-path import allowlist, so we chose accuracy by hand-checking instead) — for anyone who wants the specifics without reading source.

New events are additive, not architectural. VALID_TRANSITIONS (app/state_machine.py) is one flat set of (from_state, to_state) tuples, not scattered ad-hoc checks — a new event type (a calendar-invite response, a new input format, a new terminal outcome) is a new tuple in that one set plus one write_gate action, reviewed in one place, never a rearchitecture. Every record in the system, existing or new, gets its id the same way: app/ids.py's new_id(prefix) is not target/account-specific — it is a short, prefixed, collision-safe id generator for any entity type, so a genuinely new kind of record (an invite, an event) costs zero new ID machinery. This isn't a design promise we're asking to be taken on faith: it's the exact mechanism that let a second live vertical (ai-security — real companies, a real ICP, a real run) go from zero to end-to-end with one new YAML offer file and no new code, cited above. Adding a new input source follows the same shape — import_csv is the one deterministic ingestion path into accounts/targets; a different source format means a new parser feeding that same path, not a new write path or a new state to invent.

Real scheduling, not a placeholder link. A follow-up draft that proposes a meeting time proposes a REAL one: a scheduling agent picks a slot from a real, already-computed weekly calendar (filtered against every slot any other target has already taken), and before that reservation is ever written, deterministic code re-checks the slot is still open — the exact same verdict-then-action split the ICP judge uses. The model never gets to write a time directly; the footer states one only because code composed it after the model ran.

How we built it

Google ADK 2.7.1 (pinned exactly), Gemini 3.5 Flash via Vertex AI with ADC in deployment and an API key locally, Cloud SQL (Postgres) behind a single connect() seam that also speaks SQLite for local runs, and a FastAPI console deployed and live on Cloud Run. Structured I/O is Pydantic throughout.

Data sources: every fact the research agent uses comes from either google_search (results text only — the model never sees a rendered page) or a direct HTTP fetch of the company's own site (url_context / a direct page GET), both logged to the sources table with the URL and the raw text actually retrieved, so every claim in a signal can be checked against what was really on the page. Nothing is scraped from a third-party data broker or a paid enrichment API.

The repo replaced a LangGraph implementation with ADK during the hackathon.

Challenges we ran into

  1. It's genuinely hard to demo the whole process without disclosing the Gmail API and using a real inbox with real messages from a real company, which is rare and not realistic to set up for a demo. I ended up overriding the fetch_inbox tool call with a fake reply draft I wrote myself, and testing whether the classifier could still read it correctly, with real confidence.
  2. State transition guarantees are hard to promise in general — I had to actually think through how to make one real, not just say it's handled. That's most of what state_machine.py's VALID_TRANSITIONS table and the write-gate tests exist to prove.
  3. Gemini 3.x Flash bills its thinking tokens against the same max_output_tokens budget as the actual answer. First time I hit this, I had it set to 1024, thinking ate 979 of that, and every structured call just came back None. Took a while to figure out that's what was happening — nothing in the error told me directly.
  4. Gemini 3.x is only served from Vertex's location=global — I burned real time getting 404s from the regional endpoint before I found that out.
  5. Had a connection just hang once, mid-batch. Sat there for almost 10 hours before I noticed, and it had barely used any actual CPU the whole time. Added a real per-request timeout and a wall-clock ceiling per target so that can't happen silently again.
  6. There's a real race if an operator approves, then approves again with an edit, in the same second — the send gate was picking "the latest decision" off a timestamp that only has 1-second resolution, so which one won was arbitrary. Fixed by adding a real sequence number instead of trusting the clock. Only found it because a test passed by itself and failed inside the full suite.
  7. The natural-language agent crashed on literally every real target the first time I ran it live — asyncio.run() cannot be called from a running event loop, on every single one, before a single model call went out. Three of the stage functions call asyncio.run() internally, and the agent's own tool-calling runtime already owns an event loop on the same thread. 550+ passing tests never caught it, because they mock one layer above where the nesting actually happens. Only found it by running the real thing.
  8. Even after fixing that, one Taskmaster call still has a 600-second ceiling on the WHOLE run, not per target — I built it that way originally and didn't notice the mismatch. A big enough batch can just run out of time mid-research and leave targets stuck. Took two real batches actually failing this way, live, before I built the outer loop above that resumes instead of just raising the number.

Accomplishments that we're proud of

What we measured (real run, 10 targets):

Our LLM judge disagreed with our own deterministic scorer on 6 of 10 real targets — twice downgrading a good_fit to not_target, twice to watchlist, twice promoting a watchlist to strong_fit. Every divergence carries a written justification, required by a Pydantic validator; the judge cannot overrule the formula silently. One reads:

"The company deviates significantly from the target ICP's industry focus (mental health/therapy/counselling vs. general/specialist healthcare) and size parameters (3-20 clinicians vs. 200-500 employees), making them an inappropriate recipient for this specific pitch."

The formula had scored that company good_fit. Demoting the deterministic score from verdict to evidence changed the outcome on 60% of real targets — the architecture decision is measurable, not decorative.

Only 10% of our agent's signals were verifiable against stored source text. Of 31 signals, 3 are source tier and 28 are findings tier. Every signal must quote its evidence, and we check that quote against raw page text we persisted; google_search and url_context resolve server-side, so their text can never be captured and a claim derived from them is recorded as findings, never as verified. We are publishing the unflattering number, because a groundedness metric you only report when it looks good is not a metric.

The failures are all in the repo. docs/data-flow.md documents every one, including the ones that made us look bad. The progress tracker records which tickets were wrong and which the implementing agent caught.

Every send-gate check is satisfiable on real data now — zero placeholders, proven end to end. Earlier in the build all four of contact-email capture, email_verified, a content-policy runner, and a prompt-injection scanner were unsatisfiable on real data, and we said so publicly. We then built all four for real: operator-asserted email_verified from a CSV column with no network call (no fake verification — an honest assertion, syntax-validated); a deterministic content-policy runner and injection scanner that evaluate every persisted draft, fresh or edited, and write their own two gate columns. docs/gates.md §2.2 went from 4 unsatisfiable to 0.

Then we proved it wasn't just tests: we ran the entire pipeline live, on one real, un-preselected company we did not pick or seed in advance (MindnLife, a Hong Kong/London therapy practice). The agent's first fetch got a 403 and it recovered on its own with search-grounded research, found real evidence (a booking-system migration, an office expansion, a payment-workflow gap, limited admin hours), scored it, drafted an email, and stopped hard at the human gate. A real person approved it through the console — the same API call a click makes. The real 19-item send gate passed. A scripted reply came back positive. The real classifier called it positive at 0.98 confidence and queued a follow-up.

The judge diverges from the deterministic formula in both directions, on real data, not just one cherry-picked case. Psychotherapy Counselling Clinic scored strong_fit by the numeric formula; the judge overruled it to not_target — it's in Victoria, Australia, and our ICP is Hong Kong only, a disqualifier the formula's fields never checked. CrewAI, on the ai-security offer, scored watchlist at 58 — mostly because Phase 1 found no contact data — and the judge overruled it up to strong_fit, reasoning in its own written justification that a missing contact isn't a missing ICP fit for one of the most prominent agent-orchestration frameworks with exactly the complex OAuth/tool-use surface the offer is built to audit. Both directions require the judge to write down why, in a schema that rejects a divergence with no justification — the guardrail doesn't just catch false positives, it also catches the formula being too conservative.

Then we asked a fresh LLM with zero briefing to grade our own UI. No repo access, no docs, curl only against the running console — the same blind spot a Devpost judge has. It correctly reconstructed the whole story: what the product does, the MindnLife narrative in order, which steps were automated versus the one human decision, and — citing the simulated:true flags and the reply's reserved .test domain as its own evidence — correctly concluded nothing was ever actually sent. It self-rated 4/5 confidence. If a judge with a five-minute video and no briefing can't be expected to understand more than that cold model did, we didn't want to find out during judging.

What we learned

Mocked tests prove a function works; they don't prove a system works. 550+ passing tests never caught the natural-language interface crashing on every real target — the mocks patched one layer above where asyncio.run() actually nested inside an already-running event loop. The fix wasn't a better mock. It was running the real thing on real infrastructure before trusting the suite.

A guardrail is only real if it changes an outcome, not just if it exists. We didn't know the ICP judge mattered until we measured it disagreeing with our own formula on 6 of 10 real targets, in both directions. If it had agreed every time, "the LLM can override the score with justification" would have been an untested assumption wearing a Pydantic schema.

Publish the number that makes you look bad, or the metric isn't real. Only 10% of our signals were verifiable against stored source text — we could have called all 31 "grounded" and no test would have caught the lie. We didn't, because a groundedness metric you only report when it's flattering isn't measuring anything.

Determinism has to be the default you fall back to, not a feature you remember to add. Every near-miss in this build — the same-second approval race, the CSV re-import collision, the missing footer field — was a place where letting the model (or a human, at speed) decide "in the moment" would have been faster to write and wrong under load. The pattern that held was building the deterministic path first and only ever letting the model produce a verdict for it to act on.

Real experimentation infrastructure, deliberately not a closed learning loop. Every first-touch draft is deterministically assigned one of ten hand-written style hypotheses (tone, structure, CTA style), and scripts/hypothesis_scoreboard.py computes each one's real win/loss record straight from the reply router's own trusted verdicts — never a raw, possibly low-confidence classification, so the same P4 confidence-floor discipline that governs every other action in this system governs this measurement too. What it does NOT do — on purpose — is feed that score back into which hypothesis gets tried next: selection is a pure function of the target, unaffected by outcomes. Outcome-linked re-weighting, including the guardrails a responsible version needs (bounded weights, no drop-to-zero without a human, full audit trail), is fully specified in docs/feedback-loop.md and deliberately not built for this submission — a learning loop without its guardrails is worse than no learning loop at all. I don't have a clear way yet to guarantee an LLM-generated hypothesis would always be safe to put in a draft writer's prompt, which is why I only included 10 hand-written hypotheses and let the system test those for me — the ones with a higher win rate should get leaned on more in future drafts, and the ones that don't work should be avoided. The only role the LLM plays here is running the test: the higher the score, the more aggressively that hypothesis gets tried. That part isn't implemented yet, for the reason above — the current version is purely an experiment and a proof of concept for a self-learning agent. I don't regret the decision, though: my core thesis is safety over autonomy, and a self-improving function without proven safety isn't something I'd ship.

What's next for agentic outbound agency

Unattended reply polling via Cloud Scheduler and Pub/Sub — scoped, then deliberately cut to protect the send-side submission (it isn't required for the Google Cloud infra criterion; Cloud SQL already satisfies it). Real Gmail send, gated behind a human click in our own console — a deliberate, documented policy change from "DRY_RUN only, always" that we have not made yet. More verticals (school, clinic, one more) via the existing offer/ICP config, no new machinery needed. Deferred re-surface and reviewer learning from rejection reasons — the data (review_decisions.reason) is already captured; nothing reads it back yet.

Built with

google-adk · gemini-3.5-flash · Vertex AI · Cloud SQL (Postgres) · Cloud Run · Secret Manager · FastAPI · Pydantic · pytest · Python 3.11+

Repository

Public GitHub repo. README.md has cold-clone spin-up steps, verified from a fresh clone, and a prior-work disclosure: two commits predate the submission window, both documentation only; the first code commit is inside it.

Built With

  • cloud-run
  • cloud-sql
  • fastapi
  • gemini-3.5-flash
  • google-adk
  • pydantic
  • pytest
  • python
  • secret-manager
  • vertex-ai
Share this project:

Updates

Submission history