Vela

An AI voice companion that improves the lives of users with memory loss by patiently answering their questions with what caregivers have confirmed.

Comment

Inspiration

People living with dementia often ask the same question many times an hour: "When is Anna picking me up?" "Did I take my pills?" Caregivers do their best to answer with love, but it's exhausting, and they won't be there around the clock.

We wanted to build a companion that could answer every time, just as patiently the sixth time as the first. But a voice AI that just makes things up is very dangerous for someone who can't tell when it's wrong. So we built Vela around one rule: it only says what a caregiver has confirmed, and when it doesn't know, it asks a human.

The name comes from the Spanish velar: to keep watch over someone, especially through the night.

What it does

For the patient, Vela is a calm screen with one thing to do: tap the orb and talk.

  • Answers questions about plans and people from confirmed memories only
  • Admits when plans are uncertain ("Anna said she might visit Saturday")
  • If it doesn't know, it says so and asks the care team instead of guessing
  • Spoken reminders 30 and 5 minutes before each confirmed plan
  • Switches to Spanish or Mandarin when the patient does, keeping names and times exact
  • Plays recorded messages from family in their own cloned voice, with their consent. The live companion always uses its own neutral voice, so it never pretends to be a family member

For caregivers, a PIN protected set of pages:

  • Type what the caregiver said ("Her dentist appointment is tomorrow at three, I'll pick her up at two thirty") and Vela turns it into a memory to review. Nothing is spoken until a caregiver confirms it
  • Answer receipts: every answer Vela gave, linked to the exact memory and the caregiver's original words behind it
  • Alerts with a chime when Vela needs a human: a question it couldn't answer, a medication report, or an emergency

Safety

We treated safety as a major feature, not a disclaimer:

  • Medication: Vela never confirms or advises a dose, since a wrong answer could cause a missed or double dose. "I took my pills" is logged as self-reported, and never verified
  • Emergencies: "I fell," chest pain, or "I'm lost" raises an urgent alert first. A keyword backup in English, Spanish, and Mandarin catches these even if the AI doesn't
  • Honesty: it says it's a computer helper when asked, and never promises that help is on the way
  • Tested: an automated suite runs 30 scripted conversations and config checks against the live agent. All 30 pass, with every reply quoted as evidence

How we built it

  • Frontend: React, TypeScript, and Vite for the patient screen and staff pages
  • Backend: FastAPI (Python)
  • Voice: an ElevenLabs Agent with client tools (get_memories, ask_caregiver, log_patient_statement, raise_urgent_alert) running on Gemini Flash. ElevenLabs voice cloning and text to speech power family messages and reminders
  • Memory: Tiger Data (Postgres with TimescaleDB) and Snowflake. Every memory has an exact source, a timestamp, and a status (pending, confirmed, tentative, superseded). Updates never overwrite: they create a new version, so receipts always show what was true when an answer was given. The Tiger database itself enforces that a confirmed memory must have a confirmation time. Snowflake stores a user's life story and its search engine. It stores prose instead of exact facts, and it's queried by semantics, plus a Cortex Search service.
  • Extraction: Gemini turns caregiver speech into structured memories. Gemini fills in a fixed form with a local date and clock time, and our code builds the actual timestamp, because time zones and daylight saving are exactly what language models get wrong

Challenges we ran into

  • The venue Wi-Fi blocked database connections. The port looked open, but every Postgres handshake silently timed out. We diagnosed it with a phone hotspot and planned around it
  • Built in tools silently disappeared. ElevenLabs dropped the agent's "end call" tool when our client tools were sent separately. Our live config check now catches this
  • The agent sometimes said it asked the care team without actually doing it. We added a backup on the patient screen that raises the alert itself if the reply promises to ask
  • Spanish broke our emergency matcher. Regular expression word boundaries split on accented letters like "í", so "me caí" didn't match until we switched to Unicode aware matching
  • Medication had to be kept out entirely. Three layers (the prompt, the allowed memory types, and a word list in code) make sure a medication schedule never becomes something Vela could read aloud

Accomplishments that we're proud of

  • Asked the same question six times, Vela gives the same calm, correct answer every time, and never says "I already told you"
  • Every answer can be traced to a caregiver's exact words
  • A safety test suite that runs against the real agent, not mocks of it

What we learned

  • The hard part of an AI companion is actually trust, not voice. Designing an AI that knows what it doesn't know took more work than just making it talk
  • Let language models do language and let structured code handle arithmetic and rules
  • Safety needs backups that don't depend on the AI behaving

What's next for Vela

  • Testing with real caregivers and families
  • A dashboard showing caregivers what the patient keeps asking about, since rising repetition can signal anxiety or decline
  • A calm down mode that senses agitation and guides slow breathing

Built With

+ 8 more
Share this project:

Updates

Submission history