Hallo

Weekly report2
Elder's page
weekly report3
Converstation recap
Elder page 2
Alerts
Weekly overview
Elder's stories
Symptoms Records
What's on the mind
Weekly report
Overview2
Use this to try out our demo

Prompt G: AI for the Aging Population By 2030, one in five Americans will be over 65, and there's nowhere near enough people to take care of everyone. The US is projected to have millions of unfilled caregiving jobs within the decade, and 53 million family members are already doing this work unpaid. Meanwhile, almost no technology is actually built for older people. Even Alexa and Google Home are frustrating for most seniors to use. AI finally makes a new class of products possible: voice interfaces that can hold real conversations, monitoring that helps older adults stay safe and independent, robotics that can assist with physical tasks around the home, and software that helps family caregivers coordinate care, appointments, and emergencies. This is one of the largest, most underserved markets in the world, and it's growing every single day.

Inspiration / What it does

Hallo is an AI companion for older adults. Part of the inspiration was Doubao, an AI that many elderly people in China already talk to every day. Their children are often busy, and long-term partners can fall out of the habit of really talking to each other. Hallo isn't meant to replace time with family. It gives elderly users a sense of connection, someone to share their thoughts with, and support when no one else is around.

Hallo has two sides:

For the elder: a voice-first companion

Hold to talk, hear the reply read aloud. There's no typing and no menus.
It remembers. Mention "I'll repot my orchid this week" and a few days later Hallo asks how the orchid is doing.
Gentle reminders. Family can set a reminder, like a daily pill or "did you book the eye doctor?". Hallo brings it up naturally in the next chat, using the family's own wording, and never adds medical advice.
Privacy the parent controls. Saying "keep this between us" hides that part from every family view, and Hallo still remembers it. Urgent safety signals (a fall, chest pain) always go through, and the parent is told this up front.

For the family: a care dashboard

A daily summary, plus a symptom timeline built automatically from casual conversation. Red-flag symptoms trigger real-time alerts.
Fall detection from a camera feed, with live push alerts.
An "on their mind" care list, sibling sharing ("I'll handle this"), and 7-day reminder adherence.
A printable weekly report, a one-page summary for the doctor, and a memoir built from their stories.

Hallo does not provide medical diagnosis.

How we built it

Backend: Python 3.12 + FastAPI, with SQLite via SQLAlchemy and Jinja2 server-rendered pages. Server-Sent Events push alerts to the family dashboard in real time.
Voice pipeline: The browser records with MediaRecorder. Audio goes through OpenAI gpt-4o-transcribe (speech to text), then gpt-4.1-mini (the reply), then gpt-4o-mini-tts (the spoken answer). The text comes back first and the audio loads right after, so a full round trip takes about 3–4s.
Understanding conversations: After each message, a background task uses OpenAI Structured Outputs with a strict schema to extract symptoms, memories and privacy requests. The alert rules (what counts as a red flag, what bypasses privacy) are plain code, not LLM decisions.
Fall detection: YOLO11-pose (Ultralytics + OpenCV) with pose-based rules, packaged as an MCP server (fall-mcp). It can be used from our dashboard agent or from any MCP client, such as Claude Desktop or Cursor.
Quality: 300+ automated tests plus two LLM eval sets, one for symptom extraction and one for memory and privacy requests. A privacy leak test scans every family-facing endpoint for private text. A mock LLM mode lets the whole demo run offline.
Tooling: uv for dependencies, pytest, and GitHub, with two developers working in parallel against an API contract we froze on day one.

Challenges we ran into

Deciding what to build in 2 days. We had far more ideas than time, so we wrote a spec first, froze the interface between the chat side and the fall-detection side, and cut anything that wasn't core to the demo.
Making voice feel natural. The transcription model invents sentences when it hears silence, so we measure the mic level in the browser and never upload a silent clip. Chinese speech sometimes came back in Traditional characters until we added a prompt hint. We also chose a sequential pipeline over the Realtime API to keep it reliable.
Keeping the companion warm instead of clinical. Telling the model to "ask one question at a time" was ignored about half the time in Chinese, so we enforce it in post-processing code. Reminders and follow-ups are limited to 3 per greeting so the chat never turns into a checklist.
Privacy vs. safety. Deciding what a parent can hide from their kids, and what must always reach them, was the hardest design question. We settled on one privacy read layer, a fixed safety bypass, and telling the parent the rule up front.
Fall detection without false alarms. We tuned the pose rules on real fall datasets. On our demo clips it caught 2/2 falls with 0/4 false positives, and on the public URFD clips it caught 4/5 falls with 0/12 false positives.

Accomplishments that we're proud of

The chatbot actually feels like a companion. It remembers small details and follows up days later ("Did you repot the orchid?").
We shipped a lot in about 2 days: voice chat, symptom logging, memory, reminders, privacy controls, fall detection, a live family dashboard, and three kinds of reports, across 120+ commits.
Privacy we can prove. An automated test checks that private content never leaks into any of the 11 family pages and APIs, while real emergencies still get through.
Fall detection as a reusable MCP server that other AI tools can plug into.

What we learned

Splitting work well is half the battle. Freezing the API contract early meant the two halves of the app fit together on the first try.
LLMs are great at understanding, and code is better at rules. We let the model extract meaning, and hard-coded anything safety-critical.
Designing for elderly users means removing things: fewer buttons, shorter sentences, one question at a time, and respect for their privacy even from their own children.
Test AI features like any other code. Small eval sets caught regressions that manual testing would have missed.

What's next for Hallo

We want to bring Hallo to iOS as a full app, and integrate with home monitoring cameras like Yi Home for real in-home fall detection. Longer term, we'd love to make Hallo's memory portable into physical robots, so the same conversations and context could help elderly users physically and give Hallo an embodied presence beyond conversation.