EmotiArm - Emotional Support Robot

A robot arm that comforts when you're sad, slaps when you're distracted

Comment

Inspiration

People working alone often lack physical feedback for emotional regulation and accountability. Digital notifications are easy to ignore, but a physical robotic companion creates real-world presence and connection.

We wanted to explore how robots can provide emotional support through touch and voice.

What it does

EmotiArm uses facial recognition to detect your emotional state in real time and responds with physical gestures and voice.

😢 SAD State
The arm gently pats your head and wipes your tears while speaking words of comfort.
“Hey. I'm here. You're going to be okay.”

😵 DISTRACTED State
The arm delivers a light slap to refocus you while giving accountability prompts.
“Focus! Eyes on the screen!”

😌 NEUTRAL State
The arm rests peacefully at your side.

All movements run on Cyberwave's digital twin simulation of the SO-101 6-axis robotic arm in real time.

How we built it

  • MediaPipe Face Mesh analyzes 468 facial landmarks to classify emotions based on mouth position, head tilt, and face direction
  • Cyberwave SDK controls the SO-101 digital twin through a cloud API with real-time joint commands
  • ElevenLabs TTS generates natural voice responses with emotional tone
  • Python + OpenCV run real-time webcam processing in a 10 Hz detection loop
  • Motion design created four keyframe sequences (REST, PAT, WIPE, SLAP) with 15+ joint positions

Challenges we ran into

  • macOS camera permissions initially blocked webcam access
  • MediaPipe version 0.10.32 broke the API and required downgrading to 0.10.8
  • Python 3.11 lacked root certificates for Cyberwave API calls, causing SSL errors
  • Getting robotic gestures to feel natural required multiple motion iterations
  • The system had to be built within a 6.5-hour coding window while debugging APIs

Accomplishments that we're proud of

  • Built a full emotion detection pipeline from scratch in under 6 hours
  • Integrated Cyberwave, ElevenLabs, and MediaPipe into one working system
  • Designed robotic motions that communicate empathy through movement
  • Implemented a fallback keyboard mode when webcam detection failed
  • Achieved real-time synchronization between robotic motion and voice synthesis

What we learned

  • Facial landmark geometry can classify basic emotions using simple logic without heavy ML models
  • Digital twins allow rapid prototyping of robotic behaviors without needing physical hardware
  • Cloud-controlled robotics opens new possibilities for remote emotional support
  • Physical presence matters. A robot reacting to you feels different than a notification
  • Hackathon lesson: always build a fallback demo mode

What's next for EmotiArm - Emotional Support Robot

Physical deployment
Deploy the system on real SO-101 hardware for actual physical interaction.

Emotion intensity detection
Detect emotional levels such as slight frustration versus crying to trigger proportional responses.

Multi-user support
Add face recognition so the robot remembers users and preferences.

App integrations
Trigger interactions based on Slack status, calendar stress, or productivity signals.

Custom motion designer
Allow users to design their own comfort gestures through a simple web interface.

Behavior learning
Track patterns over time. Example insight: “You tend to get distracted around 3 PM every day.”

Built With

  • computer-vision
  • cyberwave
  • elevenlabs
  • mediapipe
  • opencv
  • python
  • robotics
  • tts
Share this project:

Updates