HALO | Devpost

Patient dashboard
Dashboard calender
Glasses UI
Upcoming appointments
User menu to different pages

Inspiration

Shreyaan's relative suffered from dementia, and he firsthand witnessed how the condition can rob a person of their independence. His relative would often ask the same questions a dozen times in one afternoon - "Is my son here today?", "Have I taken my medication?", "Who am I meeting today?". Each time, a little more embarrassment, a little less willingness to ask. Multiply that by 55 million people worldwide and 11M+ unpaid family caregivers in the US, and the scale of the problem is people quietly losing the confidence to run their own lives.

We built HALO for the moment that a question gets asked: a display that answers without judgment, remembers when patients can't. It does not replace the family, but helps patients be confident again.

What it does

HALO is an AI care team for someone living with dementia, delivered through Even Realities G2 smart glasses and a family web portal.

When consent is given, the glasses listen to everyday conversation. They show who the person is talking to and their relationship to them, what's planned for the day, and reminders for their medication. Patients can schedule events and reminders with their voice.

The family gets a care portal, showing scheduled events and reminders as well as medications left. HALO can reorder a prescription after family members approve. The doctor uploads a prescription; HALO reads it, and the doctor confirms every field before it becomes a reminder.

It is also able to translate between Hindi and English in real-time.

How we built it

Glasses app (Even Hub SDK + Vite): microphone audio runs through a hand-built utterance segmenter (RMS-based voice activity detection) and only finished utterances are uploaded, in spoken order — so "I'll come Saturday at 2" always reaches the server before the "yes" that follows it. The 576×288 monochrome display is driven by pure screen-builder functions, tested without hardware.

Portal server: Groq Whisper for transcription, an LLM (with a strict "extract, never guess" prompt) plus chrono-node date parsing for plan extraction — the model only classifies, code verifies the date/time actually appears in the sentence, so it can't fabricate a plan. Another vision call reads uploaded prescriptions into structured dose times, quantities, and refills, all confirmed by the doctor.

Care portal + patient page: role-based views, live dose logging, pill counts, and Browser Use automation that shops a refill on GoodRx — capped at $1 per lookup — and waits for family approval before any order.

All of these were written in TypeScript.

Challenges we ran into

Making the numerous APIs work, fixing speech recognition latency, making the glasses UI more user-friendly. Small size of screen and making everything look decluttered.

Accomplishments that we're proud of

Scheduling an event, confirming it through voice alone and it being reflected on the family portal.
iMessage integration, allowing people to text using their voice and being able to ask questions to the HALO AI agent via text.
Getting real-time medication prices through GoodRx prices for the patient's ZIP, gated behind family approval and a spending limit.

What we learned

Ease-of-use, especially for unconventional/newer technologies marketed for a different demographic.

What's next for HALO

Weekly reports to Doctors and daily reports to family members for patients that might live away from family or are traveling. We also want to scale this for millions of users facing dementia.