OLAF - Elderly Care Companion

Inspiration

Most elderly people living alone get a weekly phone call. That is not enough. We built OLAF for the people who deserve more than a weekly check-in. A daily companion that listens, remembers, and quietly keeps their family informed.

What it does

OLAF is a real-time voice companion for elderly users built on Gemini Live API. It greets the user, asks how they are feeling, sets reminders, tracks mood and health, and preserves spoken memories as illustrated story chapters. Family members see a live dashboard with health trends, pending reminders, and AI-generated conversation summaries.

How we built it

The backend runs on Google Cloud Run using Google ADK with StreamingMode.BIDI. The companion agent uses gemini-2.5-flash-native-audio-preview for real-time audio. Memory illustrations are generated by Vertex AI Imagen 3 and stored in Cloud Storage. User data lives in Firestore. The frontend is a Next.js PWA hosted on Firebase Hosting, communicating with the backend over WebSocket.

Challenges we ran into

Getting the model to speak first without a double response after tool calls was the hardest problem. Gemini Live API in bidi mode fires audio before and after tool execution. We solved it by restructuring the session start trigger and tuning the system instruction so the model acknowledges the tool call in one turn only.

Accomplishments that we're proud of

The session start flow. OLAF greets the user within two seconds of connecting at the right time of day, with a reference to something from their memory bank. It feels like a real relationship, not a voice assistant.

What we learned

ADK bidi-streaming requires the backend to proxy all audio, which adds a relay hop but unlocks full server-side tool execution with Firestore context. The tradeoff is worth it. We also learned that elderly-first UX is not just larger text. It is slower pacing, warmer language, and never making the user feel like they are using a computer.

What's next for OLAF

Medication reminders with photo verification, weekly family email digests, and a caregiver alert system that escalates based on multi-day mood trends rather than single signals.

Built With

• fastapi
• firebase
• gcp
• gemini
• google-adk
• imagen
• next.js
• python
• react
• tailwind
• typescript