Calmify

Detect and Treat Overstimulation with the Click of a Doll

Comment

Inspiration + Goal

Overstimulation can make the life of a child with autism very difficult. Whether it's caused by a loud environment, crowded space, sudden changes in surroundings, or any other factor, these annoyances are amplified because of children with autism's hypersensitivity. This means that these stimuli are processed more intensely by their brains.

The thing is, many of these sensitive situations can be avoided. With just a bit of precaution. And if they do occur, their effect can be minimized.

So, I created Calmify. With the exact goal that its name emphasizes: to help these children undo the sensory overload, and reach a calmer state.

What it does

The two most important physiological signals for detecting sensory overload are pulse rate and breathing rate. I built Calmify, a software + hardware assistant that measures these vital signals. With these signals, it A) determines whether the child displayed reactions typical of a sensory overload and B) provides a multi-step, universal approach for easing the pain and bring the child back to calmness.

An Android app uses the phone's camera to monitor pulse and breathing rate in real time. When vitals spike above the child's personal baseline, the system triggers an alert to quickly warn that there may be a sensory overload.

I implemented a three-step procedure to undo the sensory overload:

1. Squeezing or applying pressure

Squeezing or applying pressure helps many autistic children calm down because it activates a sensory system called proprioception. This system tells the brain about body position, muscle tension, and pressure on the body.

As a result, the child is given a stuffed animal (filled with sensors), and their only task is to firmly grip and squeeze this animal a few times, allowing for their sensory capabilities can be slightly rejuvenated. Kids love stuffed animals, so why not play with one while helping the brain reach its best state? The strength they use is measured to ensure recovery is complete.

2. Listening to a calm voice

Listening to a calm voice helps regulate the autonomic nervous system by activating the parasympathetic rest-and-digest response, which can slow heart rate and breathing during stress. Predictable, soothing auditory input also helps the brain focus on a single signal, reducing sensory overload and supporting emotional regulation.

So, I added a voice-over that gives reassuring, calming messages to the child after the overload.

3. Breathing exercises

Breathing exercises slow the breathing rate and stimulate the parasympathetic nervous system via the vagus nerve, which helps lower heart rate and reduce the body’s stress response. Controlled breathing also restores a steady respiratory rhythm, which can stabilize physiological signals disrupted during sensory overload.

After all this, the child is presented with an AI assistant + agent, with the ability to converse with them to find root causes of the stimulation (such as "something being on their mind") and address this cause (such as automatic e-consults).

Full Pipeline

  1. Detecting Breathing Rate and Pulse Abnormalities (Presage)
  2. Triggering Overload Response (Presage + Flask)
  3. Squeezing/Pressure Activity (Arduino Gyro Sensor)
  4. Relaxing Voice (ElevenLabs)
  5. Breathing Activity (Presage)
  6. AI Assistant Conversation (ElevenLabs)
  7. Final thoughts, Recommendations, E-Consult (ElevenLabs Agent)

How I built it

Presage SmartSpectra SDK

  • Android app to extract pulse rate and breathing rate via camera (using facial blood flow patterns)
  • Fully customized UI on the Android interface, developed in Kotlin
  • Personal baseline system: 8 initial samples to establish the child's resting vitals, then compared with dynamic values (pulse +8 bpm or breathing +3.5 above baseline for 3 consecutive readings)
  • When overstimulation is detected, a warning tone plays and the SmartSpectra app sends an HTTP POST with the child's vitals to the Flask backend
  • Breathing exercises, telling user when to inhale and exhale, with the goal of easing their breathing patterns and bringing them back to calmness after the overload occurred

Presage was essential to Calmify’s design philosophy. Many children with autism dislike invasive wearables or devices touching their bodies, especially during moments of sensory overload. By using Presage’s camera-based vital sensing, Calmify can measure pulse and breathing without requiring the child to wear anything at all.

This allowed the entire system to remain non-intrusive, accessible, and comforting. Instead of adding another device to manage, Calmify simply uses a camera and software to detect stress signals and trigger calming interventions through the plush toy. Presage made it possible to build a system that respects the child’s sensory needs while still capturing meaningful physiological signals.

Arduino

  • Placed Arduino into stuffed animal to create plushie that can measure the child's grip strength
  • Programmed an Arduino Nano 33 BLE Sense with an LSM9DS1 IMU to detect squeeze gestures on a stuffed animal
  • Used accelerometer micro-vibrations: when the child squeezes, tiny rapid changes in acceleration magnitude are detected
  • The sketch counts shake events within a 500ms window and triggers when 8+ micro-vibration events occur, printing: SQUEEZE / GRIP DETECTED
  • A Python serial bridge script reads the Arduino's serial output and forwards each detected grip as a JSON payload to the Flask backend

ElevenLabs

  • ElevenLabs Text-to-Speech API to generate a calming voice message "Great job! You did really well. Take a nice, slow deep breath in..."
  • ElevenLabs Music Generation API to create soothing instrumental piano music
  • Created an ElevenLabs Conversational AI Agent (GPT-4o-mini backed) that speaks to the child after the grounding exercise, asks what caused the overstimulation, listens via speech-to-text, and validates their feelings
  • ElevenLabs Agent to book therapist appointments and give recommendations

Backend

Flask

Frontend

Wireframing with Figma, Programmed with Vanilla

Challenges I ran into

Trying to get Presage to work. The SDK was a learning experience.

Accomplishments that I'm proud of

Getting it to work. Also doing my first solo hackathon. As a child, my best friends were my stuffed animals. To really revolutionize the way we use them, not just as a toy but as a way to train fine motor skills via a slight gamification, gives me tons of hope for children's health in the future.

What I learned

Our goal on this planet is to improve the world. My way of doing this is keeping people happy and healthy. To make fun things healthy, and to make healthy things fun. When you build for passion and to learn, you end up building something incredible--like I just did. Build for yourself and build for others!

What's next for Calmify

Honestly, if I had even a little more time, Calmify could be even more amazing. Some things I would like to add and improve are...

  • Presage physiological waveforms, such as respiratory line length
  • Implement reinforcement learning systems
  • More advanced sensor systems
  • More toys to incorporate it into!
  • Smarter agent and therapy integration, maybe even with EMRs

Built With

Share this project:

Updates