Inspiration
I built Sylvia AI Agent because I wanted an AI assistant that could do more than simply answer questions.
Traditional chatbots are good at generating text and giving suggestions, but real work often requires multiple steps: understanding a goal, deciding what matters, planning the work, checking external information, taking an action, and knowing whether that action actually succeeded.
I wanted Sylvia to act as a collaborative digital operator rather than just a chatbot.
The idea was to create an AI system that understands the user's goals and preferences, coordinates specialized agents, works with real tools, and keeps the human in control whenever an important action is about to happen.
That became the foundation of Sylvia: reason, coordinate, act, verify.
What it does
Sylvia is a multi-agent AI command center designed to help organize and execute real-world work.
At the center is a Root Agent that coordinates specialized capabilities including:
- Context Analysis — understands the user's request and existing context.
- Decision Partner — considers goals, priorities, constraints, preferences, and decision rules.
- Action Planner — transforms high-level goals into ordered, actionable steps.
- Workspace Specialist — works with connected Google Workspace services such as Gmail and Google Calendar.
- Mission Control — tracks missions, steps, progress, status, blockers, and next actions.
- Decision DNA — maintains a structured representation of how the user wants decisions to be handled.
- Memory and Diagnostics — provides context continuity and visibility into system activity.
Sylvia can work with real Gmail data, identify messages requiring attention, inspect real messages, prepare replies, and request human approval before performing Gmail write operations.
The system also supports human-in-the-loop execution. Sylvia does not silently perform consequential actions. Instead, it presents the proposed action and waits for explicit approval.
For example, a Gmail workflow can be:
- Find a real email.
- Inspect the actual message.
- Prepare a reply.
- Display the proposed recipient, subject, and body.
- Request human approval.
- Continue the real ADK task after approval.
- Verify the actual Gmail tool result.
- Only report success when a genuine Gmail result is returned.
The application also includes a Mission Control interface, Decision DNA panel, Memory panel, Workspace dashboard, activity feed, voice interface, specialist visualization, and system diagnostics.
How we built it
Sylvia uses a React + TypeScript + Vite frontend connected to a production AI backend built with Google Agent Development Kit (ADK).
The frontend communicates with the backend through an A2A JSON-RPC gateway, allowing the system to maintain conversational context while interacting with the agent system.
The backend is deployed on Google Cloud Run
For Google Workspace operations, Sylvia uses the Gmail and Google Calendar APIs. OAuth credentials are protected using Google Cloud Secret Manager, while the Cloud Run service uses its service account to access the required secret.
The architecture separates reasoning from execution:
User → Sylvia Root Agent → Specialist Agent → Tool → Real External Service → Verified Result
This separation became especially important for write operations.
For Gmail draft creation, Sylvia uses an ADK confirmation flow. The frontend detects the adk_request_confirmation event, displays a human approval card, and sends the confirmation back to the active ADK task.
The frontend then parses the actual structured tool response rather than trusting natural-language claims.
This means a message saying “the draft was created” is not enough. A successful Gmail result must contain the appropriate success information, including a real Gmail draft identifier, before the interface treats the operation as successful.
I also added regression testing around the critical states of the workflow, including pending approval, successful draft creation, tool failure, prose-only claims, verified results, and successful results where independent verification is unavailable.
Challenges we ran into
One of the biggest challenges was making the system behave correctly across the boundary between the frontend and the production ADK backend.
The AI response is not always a simple text response. A single operation can contain function calls, function responses, confirmation requests, task status changes, and structured metadata.
The frontend therefore needed to understand the actual ADK response structure rather than treating everything as ordinary assistant text.
Another challenge was Google Workspace authentication in production.
The system worked locally, but the first Cloud Run deployment failed because the deployed container was missing the Google API Python client dependency required by the Gmail and Calendar integration.
After diagnosing the Cloud Run logs, I added the missing dependency, rebuilt the container, redeployed it, and verified the production service.
A further challenge was securely making the user's Google OAuth credentials available to Cloud Run without placing the token directly into the source code. This led to the use of **Google Cloud Secret Manager
The Gmail approval workflow was another difficult part.
The first implementation could display a successful-looking UI even when the backend had not actually created the draft. I changed the system so that the UI distinguishes between:
- proposed action,
- waiting for approval,
- failed operation,
- successful tool execution,
- and verified successful Gmail draft creation.
This forced the project to become much stricter about what counts as a successful action.
Accomplishments that we're proud of
We are proud that Sylvia moved beyond being a conversational demo into a system capable of interacting with real external services
The production backend successfully authenticated with the connected Google Workspace account and retrieved real Gmail messages.
Sylvia was also successfully tested against the production Gmail integration with a real message and a real human approval continuation.
The final system includes:
- A multi-agent architecture using Google ADK.
- Production deployment on Google Cloud Run.
- Real Gmail and Google Calendar integration.
- Secure OAuth credential handling through Google Cloud Secret Manager.
- A2A JSON-RPC communication between the frontend and backend.
- Mission Control for structured task execution.
- Decision DNA for personalized decision behavior.
- Human-in-the-loop approval for consequential Workspace actions.
- Structured result verification instead of relying on AI-generated success claims.
- Automated regression tests for critical frontend/backend integration states.
- Live diagnostics for monitoring the agent and Workspace integration.
Most importantly, Sylvia is designed around truthful execution status.
The interface should never claim that an external action happened merely because the model said it happened.
What we learned
The biggest lesson was that building a real AI agent system is much more than building a good prompt.
The model is only one part of the system.
Reliable agentic software also requires:
state management + tool execution + authentication + confirmation + verification + error handling
We learned that multi-agent systems become much more useful when each specialist has a clearly defined responsibility and a central agent coordinates them.
We also learned that external integrations require stronger guarantees than ordinary conversational responses.
For example, a sentence such as:
“Your Gmail draft was created.”
is not evidence by itself.
The system must check the actual tool result.
This changed how we designed Sylvia's UI and backend integration. Instead of presenting the AI's statement as truth, the application uses structured backend events and tool results as the source of truth.
We also learned how valuable human-in-the-loop design is. Automation does not have to mean removing the user from the process. Sylvia can prepare the work while keeping the user in control of consequential actions.
What's next for Sylvia Ai Agent
Sylvia is designed to become a broader **personal and business operations agent The next phase is to expand the number of connected services and specialist agents so Sylvia can manage more of the user's digital workflow from one command center.
Future directions include deeper Google Workspace integration, richer long-term memory, more advanced mission execution, additional business-operation specialists, stronger document and file management, automated follow-ups, and improved voice interaction.
The longer-term vision is for Sylvia to become an AI operating partner that can take a high-level objective such as:
Get my business organized this week.
and turn it into a structured mission, coordinate the required specialists, gather information from connected systems, ask for approval when necessary, execute approved actions, and report verified results.
The goal is simple:
Sylvia should not just tell you what to do. Sylvia should help you understand it, plan it, execute it, and verify that it actually happened.
Built With
- a2a
- agent-memory
- agentic-ai
- ai-agents
- ai-assistant
- cloud-run
- gemini
- generative-ai
- gmail-api
- google-adk
- google-calendar-api
- google-cloud
- human-in-the-loop
- mission
- multi-agent-systems
- oauth-2.0
- react
- secret-manager
- typescript
- vertex-ai
- vite
Log in or sign up for Devpost to join the conversation.