Livia, a collaborative agent for isolated seniors

Livia is a voice-first companion that simplifies daily coordination, protects against online threats, and offers empathetic, personalized interaction using selective long-term memory.

Comment

Inspiration

Having watched our own aging parents face the daily frictions of growing older alone, we experienced firsthand the anxiety that builds around managing complex paperwork, coordinating schedules across family members, and navigating digital channels where online scams are rampant. We saw how existing technology often alienated them — demanding complex navigation across screens or offering sterile, reactive chatbots that lack empathy and context.

We created Livia to give our parents — and millions of isolated seniors like them — a warm, proactive partner that protects their independence, preserves their dignity, and restores peace of mind for their families.

To help you complete your submission write-up for Devpost (or a GitHub README), here is a structured proposal based on your implementation, video screencasts, and project roadmap.

What it does

Livia acts as a collaborative, empathetic partner that combines autonomous background automation with a natural, conversational voice interface:

  • Autonomous Morning Briefing: Long before the user wakes up (at 07:30 AM), a background worker scans incoming emails, filters scam/phishing threats, extracts bill due dates into Google Calendar, and compiles a gentle daily brief with weather, cultural Saint of the day, and family news.

  • Voice-First Daily Rituals: Powered by bidirectional native audio, Livia opens each morning conversation using her pre-computed brief, helping anchor the senior's day through weather guidance, evening TV schedules, and clear administrative answers using trusted sources like service-public.fr.

  • Selective Contextual Memory: Rather than using heavy RAG overhead, Livia intelligently captures emotional and personal facts (like pet names, hobbies, or family memories) directly into Firestore during live conversation, weaving them naturally into future turns.

  • Family & Emergency Loop: Livia alerts family members if scam attempts are intercepted and provides one-tap emergency contact surfacing if the user expresses physical distress.

How we built it

Livia is built on a resilient, multi-agent architecture deployed serverless on Google Cloud:

  • Orchestrator & Specialized Sub-Agents: Built with the Google Agent Development Kit (ADK) and FastAPI, Livia's main persona delegates domain-specific tasks to specialized agents for Gmail (communication_agent) and Google Calendar (schedule_agent).

  • Models & Live Audio Runtime: Uses Gemini 3.5 Flash for text, orchestrating, and background decision-making, alongside the Gemini Live API (gemini-2.5-flash-native-audio-latest) over an AudioWorklet recorder/player interface for real-time, low-latency voice interactions.

  • State & Asynchronous Processing: Persistent storage (memories, daily briefs, user profiles, OAuth credentials) is handled via Google Cloud Firestore. An autonomous worker runs on Google Cloud Run triggered by Cloud Scheduler endpoints.

  • Frontend Surface: A clean, high-contrast web UI (base font ≥20px, large touch targets) built with Vanilla JS + Alpine.js, featuring SSE streaming for text chat and a dedicated /voice page for bidirectional audio.

Challenges we ran into

  • Protecting Live Audio Sessions from Tool Failures: In bidirectional live voice streaming, an unhandled exception inside a tool turn would tear down the entire WebSockets connection. We resolved this by creating _never_raises safety decorators that catch errors gracefully and allow Livia to inform the user verbally without crashing.

  • Handling API Rate Limits (429): High-frequency streaming calls can occasionally trigger rate limits. We implemented transparent backoff retries and engineered warm, friendly UI banners so the user never sees technical error codes.

  • Scam Protection Without Instruction Injection: Sifting through raw email bodies meant LLMs could be tricked by malicious text embedded in phishing emails. We built explicit heuristics to flag suspect messages and prevented raw email instructions from overwriting system safety prompts.

Accomplishments that we're proud of

  • True Asynchronous Autonomy: Demonstrating a real background pipeline where tasks are parsed, security threats are neutralized, and calendars are updated while the user is asleep, proving the true potential of autonomous AI agents.

  • Zero-Friction Senior UX: Crafting a voice interface that feels like a warm conversation rather than an interrogation, complete with accessible typography, patient tones, and zero complex navigation.

  • Deterministic & Safe Memory System: Achieving rich, multi-session emotional memory using clean Firestore document structures instead of brittle or expensive vector databases.

What we learned

  • Routines Drive Retention: Isolated seniors value daily anchor points (weather precautions, saint days, evening TV programs) far more than exhaustive feature lists or generic search tools.

  • Voice Demands Failure-Proof Design: When voice is the primary interface, systems cannot fail silently or throw raw tracebacks; every sub-agent and tool wrapper must be engineered for graceful degradation.

  • Privacy Boundaries are Critical: Staying strictly clear of medical data (HDS compliance) and financial execution builds deep trust with both seniors and their families.

What's next for Livia, a collaborative agent for isolated seniors

  • Family Mobile & WhatsApp Gateway: Enabling family members to drop photos, voice messages, and event updates directly into Livia's Firestore queue via a dedicated messaging bot.

  • On-Device Safety Guardrails with Gemma: Deploying a containerized Gemma model on Cloud Run to perform localized offline phishing and PII pre-filtering before passing payloads to main models.

  • Expanded Local Cultural & Social Content: Connecting with local municipal archives and public event registries to offer hyper-local news, market schedules, and cultural storytelling.

Built With

Share this project:

Updates

Submission history