Relay Agent

Turns messy handoff notes into clear, trackable responsibilities so nothing gets lost between teams.

Comment

Inspiration

Relay Agent came from a simple problem I kept thinking about in shift-based work: important tasks can get lost between teams.

One shift may notice an issue and assume the next shift will handle it. The next shift may think the first team already resolved it. Sometimes the information exists somewhere in notes, chat, or a verbal handoff, but it is unclear who owns the follow-up.

That creates a dangerous gap between “someone mentioned it” and “someone is actually responsible for it.”

I wanted to build something that focuses specifically on that gap.

The idea behind Relay is simple:

Work should not disappear just because the shift changes.

Instead of being another general task manager, Relay is designed as a continuity agent that identifies unresolved work, helps a human confirm what actually needs to carry forward, and makes the handoff explicit.

What it does

Relay Agent takes messy operational shift notes and uses Gemini to identify work that still needs attention.

From the notes, Relay can propose structured obligations such as:

  • what is still unresolved
  • who may own the follow-up
  • dependencies or blockers
  • what condition needs to be checked next
  • the original source evidence
  • a confidence score

The AI does not directly create authoritative tasks.

A human first reviews the proposed items, edits them if necessary, and approves or rejects them. Only approved items become part of the actual handoff.

From there, Relay follows a deterministic workflow:

Analyze → Review → Approve → Transfer → Acknowledge → Resolve

This makes it clear what the outgoing shift is passing forward and what the incoming shift has actually accepted responsibility for.

The current demo focuses on warehouse and operations shift handoffs, but the same idea could apply to manufacturing, IT operations, facilities, transportation, field service, and other environments where unfinished work moves between teams.

How we built it

Relay uses a Next.js and React frontend with a FastAPI Python backend.

Gemini is used to analyze raw notes and extract candidate obligations. Google Agent Development Kit (ADK) powers the AI intake agent.

One of the most important architectural choices was separating AI reasoning from authoritative workflow state.

Gemini can propose what it believes is unresolved, but it cannot directly change the handoff. The human reviews the output first, and deterministic backend logic controls the actual workflow transitions.

The stack includes:

  • Next.js
  • React
  • TypeScript
  • FastAPI
  • Python
  • Gemini
  • Google ADK
  • Pydantic
  • Google Cloud Run
  • Google Secret Manager
  • Vercel

The frontend is deployed on Vercel, while the Python API and Gemini integration run on Google Cloud Run. The Gemini API key is stored securely through Google Secret Manager.

Challenges we ran into

One of the biggest challenges was making sure the human review step was real rather than cosmetic.

At one point, Gemini correctly extracted candidate tasks and the user could review them, but the handoff creation path still used the original demo notes instead of the reviewed AI output.

That meant the interface suggested that the human was approving the AI’s recommendations, but those approvals were not actually controlling the authoritative handoff.

We changed the flow so only approved candidates now become the obligations passed forward.

Another challenge involved lifecycle rules. Tasks could initially be resolved while the handoff was still in draft state. That could complete the handoff before it was ever transferred or acknowledged by the next shift. We moved that rule into the backend so the application now enforces the intended sequence.

We also worked through AI loading states, accidental duplicate Gemini requests, CORS configuration, Cloud Run deployment, Secret Manager permissions, Python runtime compatibility, and container startup issues.

The deployment process itself became a useful lesson in how many pieces have to work together before an agentic application is actually usable outside localhost.

Accomplishments that we're proud of

I am most proud that Relay is not just an LLM generating a task list.

The system has a clear trust boundary:

Gemini proposes. Humans review. Relay commits.

The AI contribution is visible, but it does not silently become operational truth.

Other accomplishments include:

  • extracting structured obligations from messy shift notes
  • preserving source evidence for each AI-generated candidate
  • allowing humans to edit, approve, or reject AI recommendations
  • preventing unchanged notes from triggering unnecessary duplicate Gemini calls
  • showing a visible AI processing state tied to the real Gemini request
  • ensuring approved AI output becomes the authoritative handoff
  • enforcing transfer, acknowledgement, and resolution rules in the backend
  • maintaining an audit-style handoff history
  • deploying the frontend and backend separately using Vercel and Google Cloud Run
  • validating the backend flow with automated tests

What we learned

The biggest lesson was that building an agentic application is not mainly about calling an AI model.

The harder question is deciding what the AI should be allowed to do.

For Relay, Gemini is useful because human shift notes are messy and inconsistent. AI can interpret that context and identify possible unresolved obligations much faster than rigid parsing rules.

But operational state is different.

Whether a task has officially been handed to another shift should not depend entirely on probabilistic model output.

That led to Relay's core design:

Use AI for interpretation, humans for confirmation, and deterministic code for commitment.

I also learned that AI latency should not always be hidden. Showing Relay actively analyzing the notes helps users understand when the agent is doing real work instead of making the results look like static database content.

What's next for Relay Agent

The current hackathon version demonstrates the full continuity workflow within an active handoff session.

The next major step is persistent storage.

Relay should be able to remember an existing handoff across refreshes, restarts, shifts, and devices so a team can return later and continue from the same reviewed state instead of analyzing the same information again.

Future improvements include:

  • persistent handoff storage
  • shared team and shift boards
  • task claiming and explicit ownership
  • notifications for unresolved obligations
  • integrations with workplace communication tools
  • multi-team and multi-site handoffs
  • configurable workflows for different operational environments
  • stronger reporting and audit views
  • automatic detection of aging or repeatedly handed-off obligations

The broader goal is for Relay to become a continuity layer between people, teams, and AI agents — making sure unfinished work survives the handoff instead of disappearing between shifts.

Built With

Share this project:

Updates