Team Aida

Inspiration

Many of us in the team have elderly grandparents or parents who are living independently but we know would really benefit from more support and care in their day-to-day lives. It's difficult to make sure they get the level of attention they deserve as their carers are often at full capacity and our own busy lifestyles make it hard to be able to be as present as we'd like. This is what inspired us to build Aida.

Here is the video with highlights of Gautham's grandad using the tool link

What it does

Aida is an Eleven Labs-powered check-in tool designed to support both elderly individuals and their caregivers. With a simple phone call, users can easily share their needs—whether it’s groceries, companionship, or a stimulating conversation. Aida provides natural, compassionate interactions that foster connection and provide stimulation that is otherwise hard to find. For caregivers, Aida delivers real-time feedback from each call, helping them manage multiple clients more efficiently and prioritize care based on immediate needs—ensuring no request goes unnoticed.Using advanced language processing, Aida also analyzes cognitive metrics like part-of-speech (POS) diversity and speech patterns, offering deeper insights into users' well-being.

How we built it

We began by exploring ElevenLabs’ conversational AI, iterating on prompt design to create natural, engaging conversations while establishing rails to guide the check-in flow. (Romy) To enable phone calls, we integrated Twilio’s programmable voice API, ensuring seamless call initiation and management. (Steve)

For personalization, we implemented both short-term memory (capturing recent context within each call) and long-term memory (persisting summaries of past interactions in a vector database). These memory layers enrich the LLM’s knowledge base, allowing Aida to reference previous conversations and tailor responses based on each user’s unique preferences, needs, and history. ( Gautham)

Building on ElevenLabs’ Tools framework, we equipped Aida with capabilities to take actions beyond conversation. Using Perplexity’s API, Aida can fetch real-time news aligned with users' interests, while Twilio is leveraged to schedule medication reminders. Additionally, we developed a backend pipeline that logs structured call data and insights into a relational database, powering a caregiver-facing dashboard designed for real-time monitoring. (All team effort)

To unlock deeper cognitive insights, we applied advanced natural language processing (NLP) techniques on transcribed call data. By analyzing cognitive markers such as part-of-speech (POS) diversity, speech fluency, and conversational coherence, Aida provides caregivers with indicators of cognitive well-being, helping them identify subtle changes in users' mental and emotional states over time. (Dorian)

Challenges we ran into

Latency of the conversational AI that can impede the flow of the conversation
Making Aida feel natural and human

Accomplishments that we're proud of

We actually tested it in the wild on Gautham ( team member)'s grandfather who lives in India and he really loved it. He was able to have a discussion on topics that interested him, test out asking for medication reminders and we could see all the insights funnelling into the dashboard application live. This felt pretty great and made it all feel more real.

What we learned

Finding the right balance between a natural conversation and a structured check-in required careful prompt engineering. Ensuring the AI stayed on track without sounding rigid was important.
We learned that personalization isn’t just about remembering facts—it’s about using both short- and long-term memory to create interactions that feel familiar and human-like.
Analyzing linguistic patterns such as POS diversity, speech fluency, and coherence offered unexpected insights and was a new area for us to look into