AURA - The Empathetic AI

Mood Timeline & Mood-Based Conversations
Agent Execution Transparency with Raw Data
Conversational Empathy and Agent Overview
Multimedia Content Curation Based on Mood
Poetic Journal Reflections

Inspiration

We were inspired by Her, the film where an AI named Samantha doesn't just respond, but she listens, learns, and loves. Her emotional presence wasn't just futuristic, it felt intimate and believable. That vision stayed with us. Could we create something that isn’t just smart, but emotionally intelligent? Could we build an AI that feels?

From this inspiration, AURA was born. A system not built just to answer, but to care. To speak with empathy. To remember. To reflect. To support.

What it does?

AURA is a multi-agent AI companion that interacts with users through voice or text. It understands the emotional tone of each input, remembers past conversations, and responds with curated content, poetic reflections, and compassionate replies.

It doesn't stop there. AURA writes journal entries based on emotional history, uses metaphor-rich prompts to comfort users, and fetches media content tailored to how the user feels. From mood detection to multimedia curation, it delivers a seamless, emotionally engaging experience.

How we built it?

AURA is built on a custom multi-agent architecture using Python (FastAPI) and React.js. The system includes seven primary agents and bonus components, each with a clear role:

Core Agents:

Emotion Analyzer: Uses NLP models to classify mood from voice or text.
Curator Agent: Gathers music, videos, and news tailored to emotional state.
Journal Agent: Converts emotional input into poetic, timestamped journal logs.
Memory Agent: Stores session history and retrieves context for continuity.
Support Agent: Offers metaphorical, comforting responses inspired by Her.
Fallback Agent: Ensures the user always receives a meaningful reply.
Base Agent: Provides shared utilities, prompt handling, and OpenAI access.

Bonus Agents/Features:

Voice Integration (ElevenLabs): Enables spoken input and output for immersive interactions.
Real-Time Dashboard: Visualizes all agent states live via a React frontend.
Zero-shot Query Classifier: Helps the Curator Agent distinguish emotional vs factual queries.
Custom Message Protocols: Allows agents to communicate asynchronously without external orchestration tools.

We also integrated multiple APIs: OpenAI, Spotify, YouTube, Tavily, Tableau, and ElevenLabs which are all working in sync to provide voice, content, and intelligence.

Challenges we ran into

Orchestrating async agents without lag or failure was technically tricky.
Emotion tuning in responses required deep prompt engineering to avoid robotic outputs.
API integration with varied formats, tokens, and rate limits was time-consuming.
Poetic journaling logic needed iteration to maintain rhythm, clarity, and relevance.
Voice latency occasionally broke conversational flow and needed buffering optimizations.

Accomplishments that we're proud of

Built an end-to-end, emotionally aware AI system with zero orchestration libraries.
Created realistic, empathetic responses using metaphor and mood context.
Seamlessly connected voice, memory, journaling, and content into one UX.
Designed a robust fallback system ensuring uninterrupted emotional continuity.
Shaped an experience that feels less like a chatbot and more like a companion.

What we learned

Emotional design matters. The feeling behind the response changes how it’s received.
Modular agent design allows flexibility, scalability, and faster debugging.
Tone engineering with LLMs is more powerful than just prompt design; it’s emotional UX.
Building with empathy in mind makes you think differently as a creator and as a user.

What's next for AURA - The Empathetic AI

We’re excited to continue developing AURA:

Deploying to the cloud so it’s always available, everywhere.
Learning personalization, allowing AURA to adapt its replies and tone to the user over time.
Wearable integration for real-time emotion detection using heart rate, sleep data, or voice stress.
Mobile and web app extension for broader reach.

AURA began as an experiment. Now it feels like the beginning of something unbelievably real. A future where AI isn’t just smart but sincerely present.