Henry The task master

Henry is an autonomous execution agent built with Google ADK, Gemini, and Google Cloud. Users delegate outcomes—like project reviews—and Henry plans, coordinates, and verifies results end-to-end.

Comment

Inspiration

Most AI assistants are good at answering questions, but the responsibility quickly returns to the user: copy the answer, open another application, perform the action, remember to follow up, and verify the result.

We built Henry around a different interaction model: delegate outcomes, not prompts. Instead of explaining how to coordinate a meeting, Henry manages the end-to-end workflow—within clear safety limits—and interrupts the user only when a decision genuinely requires human authority.

What it does

Henry is an autonomous execution agent for work that spans multiple steps, applications, and periods of time.

A user can delegate an outcome such as:

"Coordinate a 30-minute project review before Thursday, protect my Tuesday focus block, contact the participant once, and do not spend more than $0.10."

Henry converts that request into a bounded mission $M$ defined by the tuple:

$$M = (O, A_{\text{auth}}, A_{\text{forbid}}, B, P, C)$$

Where $O$ is the objective, $A$ represents authorized and forbidden actions, $B$ is the execution budget constraint ($B \le \$0.10$), $P$ is the multi-step plan, and $C$ represents the completion criteria.

  • Execution: Inspects calendar availability, preserves protected commitments, waits for external events without background polling, requests explicit approval before creating an event, and reads the result back to satisfy $C$ before declaring success.
  • Control: The dashboard exposes Henry's complete operational state: execution path, evidence, approval requests, retries, cost ledger, guardrail usage, and durable activity history. Users can pause, resume, approve, reject, or inspect a mission at any time.

How we built it

  • Frontend: Built with React and TypeScript, leveraging Three.js and GSAP for interactive mission visualization and dynamic workflow graphs.
  • Backend: Powered by a modular FastAPI application. Google ADK and Gemini generate structured mission plans, but the model cannot control durable state directly. A deterministic mission engine owns state transitions, authorization boundaries, retries, idempotency, budgets, and completion verification.
  • Infrastructure: Production utilizes Cloud Run for the API and worker, Firestore for mission state, Cloud Tasks for authenticated workflow wake-ups, and Google Calendar via OAuth 2.0. Provider interfaces allow the engine to run on deterministic local simulators for reproducible testing.

Challenges we ran into

  • Autonomy vs. Control: Giving an agent unrestricted tool access makes a demo look powerful, but undermines system trust. We implemented strict guardrails enforcing execution step limits ($S_{\text{max}}$) and cost boundaries:

$$\sum_{t=1}^{k} \text{Cost}(t) \le B$$

  • Long-Running Workflows: Henry must pause while waiting for external inputs, preserve state integrity, and resume without duplicating actions. We resolved this using persisted mission state machines, stable action idempotency keys, and event-driven wake-ups rather than compute-heavy polling.
  • Verified Completion: Creating an event is not enough; Henry reads the event back from the API and evaluates it against the initial constraints before marking a mission complete.
  • Reliable Evaluation: To guarantee reliability during testing without requiring active credentials, deterministic providers mirror live adapter state transitions without altering the mission engine.

Accomplishments that we're proud of

We built a complete vertical workflow covering delegation, planning, calendar inspection, external waiting, recovery, human approval, action execution, and independent verification.

  • Backend Quality: 55 backend tests achieving 99% unit coverage ($\text{Coverage} = 0.99$).
  • Frontend Reliability: 25 verified workflow end-to-end tests and a complete production build.
  • Architecture: State-driven UI state synchronization ensuring frontend visuals accurately reflect underlying backend state transitions without decorative decoupling.

What we learned

  • System Architecture Over LLM Alone: A language model is only one layer of an autonomous agent. Gemini handles reasoning and plan generation, while system reliability depends entirely on deterministic state persistence, policy enforcement, idempotency, and verification loops.
  • Human Approval as First-Class State: Human authorization should be modeled as a core workflow state transition rather than embedded inside prompt text.
  • Efficiency of Event-Driven Idle States: A system that safely halts execution while waiting consumes zero redundant compute, making it drastically more cost-effective than continuous polling loops.

What's next for Henry The task master

  • Expanded Integrations: Adding multi-user Google OAuth, additional communication adapters, richer event triggers, long-term memory management, and production observability.
  • Reusable Mission Templates: Building configurable workflows for operations, recruiting, project coordination, and administrative management.
  • Long-Term Vision: Establishing a dependable execution layer that coordinates work across complex web services while remaining strictly bounded, inspectable, and accountable.

Built With

Share this project:

Updates

Submission history