Carrie - AI Therapy In Your Pocket

A Judgment Free Place to Discuss Everyday Problems.

Comment

💭 Inspiration

Here's the reality: healthcare workers, professionals, and everyday people are burning out faster than ever. Traditional therapy? That'll be $100-300 per hour, and oh, you'll need to wait 2-8 weeks for an appointment.

We watched nurses and healthcare workers struggling with their mental health, facing stigma, and reluctant to seek help because of cost, time, or judgment. We asked ourselves: what if there was something available 24/7, completely free, judgment-free, and could actually understand how you're feeling—not just what you're saying?

That question became Carrie: an AI therapy companion that sees your emotions in real-time and responds with genuine empathy. 💙

✨ What it does

Carrie is your emotion-aware AI therapy companion, ready to support you 24/7 through live video calling. Here's what makes her special:

🎭 Real-time emotion detection: Using computer vision, Carrie captures frames from your webcam every 2 seconds and analyzes your facial expressions. She can detect 7 emotions (happy, sad, angry, fear, surprise, disgust, neutral) with confidence scores, so she knows exactly how you're feeling.

💬 Emotion-aware conversations: Carrie adapts her entire approach based on your emotions. She's upbeat and encouraging when you're happy, gentle and supportive when you're sad, and calm and validating when you're angry. It's like talking to someone who truly gets you.

📹 Live video therapy sessions: Face-to-face interaction through LiveKit with natural, empathetic voice synthesis powered by ElevenLabs. It feels like a real conversation.

📊 Conversation history: Track your emotional journey over time. Look back and see how far you've come with persistent session logging.

🔐 Complete privacy: Anonymous, secure conversations with JWT authentication and encrypted data storage. Your mental health journey stays yours.

🛠️ How we built it

Carrie is a full-stack microservices powerhouse with three main components:

Frontend (React + TypeScript) 💻

  • Built with Vite, React 18.3, and TailwindCSS for a smooth, responsive UI
  • LiveKit Client SDK handles WebRTC video/audio streaming
  • Canvas API captures webcam frames, converts them to base64, and sends them to the backend every 2 seconds
  • React Query keeps everything fast with efficient data fetching and conversation history caching

Backend (Node.js + Express + TypeScript) ⚙️

  • RESTful API with Express handling authentication, emotion detection proxy, and agent orchestration
  • MongoDB Atlas stores user accounts and conversation data
  • LiveKit Server SDK generates secure video room access tokens
  • Our custom Agent Manager enriches user messages with emotion context before sending them to ElevenLabs Conversational AI
  • JWT-based authentication with bcrypt password hashing keeps everything secure

AI Service (Python + Flask) 🧠

  • OpenCV Haar Cascade detects faces from incoming video frames
  • Hugging Face Vision Transformer (ViT) model classifies emotions into 7 categories
  • Flask REST API exposes /api/process_frame, /api/emotion, and /api/webhook/emotion endpoints
  • SocketIO enables real-time emotion updates
  • Custom webhook endpoint formatted specifically for ElevenLabs to consume emotion data

DevOps & Deployment ☁️

  • Docker Compose orchestrates all three services (frontend, backend, AI)
  • AWS ECS Fargate hosts everything in production
  • AWS ECR for container registry, Secrets Manager for credentials
  • nginx serves the frontend

😅 Challenges we ran into

Finding our direction: During ideation, we went in circles trying to nail down a novel angle for the startup/entrepreneur tracks at StormHacks 2025. After several pivots, we landed on emotion-aware AI therapy—something that felt both impactful and technically exciting.

Making real-time emotion detection work: This was hard. Getting sentiment analysis to work in real-time without breaking everything required us to:

  • Pull live video frames from LiveKit without disrupting the call
  • Process frames fast enough (2-second intervals) to feel responsive
  • Balance image quality (JPEG compression at 60%, 640px max width) with upload speed
  • Handle edge cases like poor lighting or when no face is detected

Backend learning curve: Most of us came in with limited backend experience. We learned on the fly:

  • How to build a proxy layer between the frontend and Python AI service
  • How to fetch emotion data via webhooks and inject it into ElevenLabs API calls
  • How to manage multiple microservices talking to each other via HTTP and WebSocket

Prompt engineering for empathy: Getting Carrie to respond differently based on detected emotions took serious iteration. We spent hours tweaking ElevenLabs' system prompts and dynamic variables to nail the right tone for each emotional state. It had to feel genuine, not robotic.

🏆 Accomplishments that we're proud of

✅ We built a working prototype with real-time emotion detection fully integrated into live video therapy sessions—and it actually works!

✅ We achieved sub-2-second latency from webcam capture → AI processing → response generation. That's lightning fast.

✅ We created a scalable microservices architecture that can be deployed to AWS ECS Fargate with Docker. This isn't just a hackathon project—it's production-ready.

✅ We seamlessly integrated cutting-edge AI technologies: OpenCV, Hugging Face Transformers, ElevenLabs Conversational AI, and LiveKit all working together in harmony.

✅ We built comprehensive documentation—deployment guides, API docs, troubleshooting resources. Future developers (and future us) will thank us.

✅ We designed an empathetic AI persona that genuinely feels supportive and adapts to your emotions in real-time. Carrie doesn't just talk at you—she talks with you.

📚 What we learned

Real-time computer vision is no joke: Balancing accuracy, speed, and bandwidth for live video emotion detection took constant optimization. Every millisecond matters.

Microservices communication is an art: We learned how to design REST APIs, webhooks, and WebSocket connections between Node.js, Python, and external services—and actually make them work together.

ElevenLabs is powerful: We discovered how to leverage dynamic variables and system prompts to create context-aware AI responses that feel genuinely empathetic.

Video streaming with LiveKit: We dove deep into WebRTC fundamentals, room management, token generation, and reconnection logic. It's complex, but so worth it.

MongoDB for mental health apps: We figured out how to model users, conversations, messages, and sessions specifically for mental health applications with the right level of privacy and structure.

Docker & AWS are game-changers: Container orchestration, ECS Fargate, ECR, Secrets Manager, CloudWatch logging—we went from zero to deployed.

🚀 What's next for Carrie - AI Therapy In Your Pocket

Immediate improvements 🎯

  • Real-time WebSocket emotion updates instead of HTTP polling for even smoother UX
  • Emotion history timeline with beautiful data visualizations showing mood trends over time
  • Voice tone analysis to complement facial emotion detection for deeper understanding

Growth & partnerships 📈

  • Customer acquisition and brand awareness campaigns to reach people who need this
  • Strategic partnerships with ElevenLabs and security-focused companies for enhanced data protection
  • Expansion across Canada, then into the U.S. market

Advanced features 🌟

  • Multi-language support (26+ languages) to help people worldwide
  • Mobile app (React Native) for on-the-go mental health support
  • Therapist matching system connecting users with licensed professionals when they need more
  • Crisis intervention system with automatic detection and emergency resource recommendations
  • CBT/DBT structured programs and AI-powered journaling for long-term growth

Long-term vision 🌍

  • HIPAA compliance audit for use in healthcare settings
  • End-to-end encryption for maximum privacy
  • Insurance integration to make professional therapy more accessible
  • Group therapy sessions and wearable device integration for holistic mental health tracking

Our ultimate goal: Make mental health support accessible, affordable, and available to everyone, anytime, anywhere. No more barriers. No more waiting. No more suffering in silence.

⚠️ Important Note

Carrie is NOT a replacement for professional mental health care. If you or someone you know is in crisis, please reach out immediately:

  • 🆘 988 Suicide & Crisis Lifeline (US/Canada): Dial 988
  • 💬 Crisis Text Line: Text HOME to 741741
  • 🚨 Emergency Services: 911

🔗 Try it out

🏗️ Built With

  • react
  • typescript
  • node.js
  • express
  • python
  • flask
  • opencv
  • pytorch
  • huggingface-transformers
  • elevenlabs-api
  • livekit
  • mongodb
  • docker
  • aws-ecs
  • tailwindcss
  • vite
  • jwt
  • websocket
  • axios
  • bcrypt
  • cors
  • emotion-detection
  • machine-learning
  • mongoose
  • multer
  • torch
  • transformers
  • vit-model

Built With

Share this project:

Updates