Otherly

Inspiration

While volunteering with the Alzheimer association, we saw how dementia patients struggle with daily life, and caregivers lack tools for personalized support. With Sam’s expertise in AI-driven healthcare and Peter’s experience building scalable AI solutions, we created Otherly—an evolving AI twin that preserves memories, detects emotions, and provides real-time companionship.

What it does

• Learns and adapts from conversations, voice, and past experiences.
• Detects emotions and responds empathetically in real time.
• Supports dementia patients and caregivers, offering familiarity and reassurance.

How we built it

The frontend is a website that records audio and sends the speech to a Python backend. STT: OpenAI Whisper LLM: Llama-3.3-70B-Versatile TTS and Voice Cloning: Play.ht The processed audio data is sent back to the website, where it is autoplayed. The LLM's knowledge and reminders are managed by feeding information through the system prompt.

Challenges we ran into

• Developing reliable emotion recognition without misinterpretations
• Creating medical safety safeguards, especially for drug adherence

Accomplishments that we're proud of

• Built AI capable of emotion-aware responses
• Implemented guard rails to prevent unverified medical guidance.
• Created a supportive tool benefiting dementia care.

What we learned

• Responds better when AI adjusts its tone and responses based on mood.
• Reducing latency in STT, LLM, and TTS pipelines improves user experience.
• Expanding beyond the website (e.g., mobile apps, smart speakers) makes adoption easier

What's next for Otherly

• Expanding to medication adherence tracking and integration with healthcare providers.
• Get old folks using it!

Built With

• elevenlabs-/-play.ai-(voice-cloning
• groq-(llm-response)
• html-/-css-/-js-(frontend)
• python-/-bottle-(backend)
• speech-brain-(ser)
• twilio(voip)
• whisper-(speech-to-text)