HoldOn — Your Time, Back

UI image for Project

Inspiration We’ve all put off a phone call because we knew it would mean navigating menus, waiting on hold, and explaining the same problem repeatedly. Canceling a subscription or questioning a bill can take more time than it should. We built HoldOn around a simple idea: describe what you need, let an AI assistant handle the tedious parts, and stay in control of the decisions.

What it does HoldOn turns a user’s goal into a coordinated workflow. Specialized agents prepare a plan, manage the conversation, check sensitive actions, and produce a summary. The dashboard shows the task’s progress, transcript, agent activity, and approval requests. Before a sensitive action proceeds, the user can approve or decline it. Our working prototype demonstrates this workflow through scripted calls, with real local Fetch.ai agent orchestration and persistent event syncing to SpacetimeDB. We also implemented a Vapi browser calling integration; end-to-end voice testing remains pending.

How we built it We built the dashboard with React and Tailwind CSS, backed by a Python FastAPI application. Our orchestration layer uses five Fetch.ai uAgents:

Orchestrator: coordinates the workflow and routes messages.
Planner: breaks the goal into steps.
Negotiator: handles replies in the scripted demo.
Guardrail: checks sensitive actions and approval decisions.
Reporter: produces the final summary. The backend stores tasks and events in SQLite, streams dashboard updates through Server-Sent Events, and mirrors events into a private SpacetimeDB module. The mirror retries failed writes and prevents duplicate events. We added optional Claude API support for planning and reporting, plus a Vapi Web SDK integration for browser voice sessions.

Challenges we ran into One challenge was coordinating multiple agents inside the FastAPI application without disrupting its event loop. We implemented message correlation, timeouts, and cleanup so agent failures could be handled predictably. Another was making approval handling reliable. A delayed or repeated response must not authorize the wrong action, so approvals are tied to specific pending requests and default to denial when they expire. Connecting SpacetimeDB also taught us about identity ownership. Our first token belonged to a different identity than the database publisher. Publishing a new database under the correct identity resolved the issue, and we verified that queued events synced successfully. Without Twilio credits, we focused on a browser voice path for the next demo milestone.

What we learned We learned that building an agent system requires more than good prompts. Clear responsibilities, explicit state, reliable message routing, and recovery behavior matter just as much. We also learned to distinguish between an integration being configured and actually working. Our dashboard reports connection state so we can verify behavior rather than assume success. Accomplishments we’re proud of We built a working dashboard, real local multi-agent orchestration, an approval workflow, persistent task history, and a verified SpacetimeDB event mirror. The prototype makes the assistant’s progress visible and gives users a concrete way to intervene before sensitive decisions.

What’s next for HoldOn Our next milestone is completing an end-to-end Vapi browser voice demo. After that, we plan to deploy the application, register its public Agentverse interface, and validate SMS summaries and real phone calls once provider access and credits are available.