TED the Bear - Therapeutic Robot

🧸 TED the Bear – AI-Powered Therapy Robot

🔹 Inspiration

Mental health challenges are increasing, yet access to therapy remains limited. We wanted to create an AI-driven emotional support companion that provides comfort, conversation, and personalized counseling. TED the Bear is designed to hug users, recognize emotions, and generate AI-driven responses, making therapy more accessible and engaging.

🔹 What We Learned

Building TED the Bear pushed our understanding of multimodal AI, robotics, and real-time interaction systems. We explored: ✅ Emotion recognition to tailor responses ✅ Speech-to-text & text-to-speech for natural conversation ✅ AI-generated counseling with voice cloning ✅ Lip-syncing avatars for immersive interaction ✅ Motorized gestures for physical comfort

This project gave us hands-on experience in integrating hardware with AI-driven software, requiring careful synchronization of sensors, motors, databases, and APIs.

🔹 How We Built It

TED the Bear combines AI, robotics, and real-time processing to create an interactive therapy experience.

Frontend:

🔹 React + Vite

Backend:

🔹 FastAPI (CRUD functions)

AI & Lip Sync:

🔹 Gooey.AI – Real-time lip-syncing avatar 🔹 OpenCV + DeepFace – Facial & emotion recognition 🔹 Whisper AI – Speech-to-text conversion

Database:

🔹 Supabase (PostgreSQL)

Robotics Integration with AI:

🔹 Raspberry Pi 4B – Wi-Fi-connected for real-time data exchange 🔹 Microphone & Speaker – Captured user speech and played AI-generated responses 🔹 Raspberry Pi Camera – Analyzed facial expressions for personalized responses 🔹 Arduino – Controlled stepper motors & depth sensors (ultrasonic) for hugs

🔹 Challenges We Faced

🚧 Real-time processing – Ensuring smooth speech-to-text and AI-generated responses without noticeable lag. 🚧 Emotion recognition accuracy – Fine-tuning DeepFace to recognize subtle facial expressions across different lighting conditions. 🚧 Hardware synchronization – Coordinating Arduino, Raspberry Pi, and AI APIs for a seamless user experience. 🚧 Voice cloning quality – Training AI voices to sound natural and expressive while maintaining low latency.

🎯 My Role in the Project

I contributed to the full-stack development of TED the Bear, focusing on:

✅ Raspberry Pi & Arduino Integration – Stepper Motors, Ultrasonic Sensors ✅ Custom Hugging Logic & Body Language – Motorized gestures for comfort ✅ Real-time Speech Recognition & AI Response System – Ensuring seamless interaction ✅ Backend Development – Built CRUD functions with Supabase

This project was a challenging yet rewarding experience, bringing together AI, robotics, and real-time communication into a single system. It reinforced the power of human-AI interaction and opened doors for further AI-driven mental health solutions.