About Hale

Inspiration

Hale was inspired by the challenges faced by people experiencing amnesia and other memory difficulties. Important conversations, appointments, and everyday moments can be forgotten, leaving patients and their families struggling to piece together what happened. We wanted to build a companion that could preserve these moments, make everyday life easier to navigate, and help families share meaningful observations with healthcare professionals.

What We Built

Hale is an AI-powered memory companion with two connected experiences: a patient-focused mobile interface and a family dashboard.

The Patient Experience

The mobile experience is designed around making memory support as effortless as possible. It supports continuous recording and transcription, alongside custom recordings that let users preserve personal stories, thoughts, and important moments from their lives.

Patients can revisit recent conversations, explore their memories through a calendar, and ask Hale questions about their conversations, appointments, medication records, and care costs. A hands-free voice assistant, powered by ElevenLabs and Backboard AI, makes this information accessible through natural conversation. It can also provide reminders for upcoming events, with Apple Reminders integration helping users stay on top of their schedules.

Accessibility is a core part of the experience. Users can switch between light, dark, and high-contrast themes and adjust text sizes to make Hale more comfortable to use.

The Family Experience

The family dashboard gives caregivers a centralized place to understand and coordinate everyday care. It includes an overview, calendar, searchable conversations, care and tasks, people profiles, Ask Hale, and settings.

Families can explore conversation timelines, review transcripts and saved memories, identify potentially important observations, manage appointment and medication records, and coordinate care costs. Weekly conversation summaries can be exported as PDFs, helping families communicate relevant context to healthcare professionals.

AI-Powered Memory Support

Hale uses voice processing and AI to transform conversations into searchable, source-linked records. Pyannote AI and our Whisker component support voice-related processing, while Backboard powers conversational question answering and memory extraction.

Hale is designed to flag potentially significant conversations and recurring observations that may be relevant to amnesia or other memory difficulties. These observations can be organized into reports containing dates, descriptions, and supporting conversation excerpts, giving doctors additional context to evaluate. Hale does not diagnose medical conditions or replace professional medical judgment.

How We Built It

We built Hale using Next.js for the application interfaces, Clerk for authentication, and TigerData PostgreSQL for persistent data storage. A Python processing worker handles recordings and transcription workflows. Pyannote AI and Whisker support voice processing, Backboard provides AI-assisted memory features, and ElevenLabs enables spoken interactions.

We also incorporated Apple Reminders integration for everyday scheduling and developed PDF export workflows for sharing conversation summaries. Our demo environment uses fictional patient data to showcase the experience without relying on real patient records.

Challenges and What We Learned

One of our biggest challenges was bringing voice processing, AI, memory organization, and care coordination together in a single, accessible experience. Continuous mobile recording introduces additional technical challenges, including background microphone access, battery consumption, permissions, and operating-system restrictions.

We also learned that AI-generated observations must remain grounded in their original sources. A flagged conversation is not a diagnosis, and a planned appointment is not proof that it occurred. Privacy, consent, data retention, and accessible design are essential when building technology for people who may already face significant everyday challenges.

What's Next

We envision Hale becoming a trusted companion that helps people preserve their experiences, supports families in coordinating care, and gives healthcare professionals a clearer history of reported memory difficulties. By keeping important conversations and everyday details within reach, Hale aims to make navigating life with memory challenges a little less overwhelming.

Built With

• clerk
• codex
• elevenlabs
• nessieapi
• nextjs
• pyannote
• python
• tailwind
• tigerdata
• typescript