MindPulse

MindPulse App Logo
MindPulse Website Home Page
Microphone permissions to help your baby sleep

Inspiration

Some of our team members work in maternal fetal medicine research, prompting exposure to babies and young parents. A rising issue within this demographic is the lack of ability to effectively sooth these babies when they are distressed. We sought to alleviate this issue through an AI powered infant soothing assistant that learns each baby's unique responses to different sound frequencies and noise profiles. Furthermore, as developers, we constantly seeing news about smart home devices and baby monitors being hacked, or companies secretly collecting raw audio data. This is a serious breach of privacy, and a huge concern in our modern day. Especially when it comes to something as vulnerable as a sleeping baby, the idea of raw microphone feeds to the cloud didn't feel secure. Overall, we wanted to build a baby sleep companion that was actually smart, but strictly local and private by design. MindPulse was developed out of the idea that we can use modern web capabilities to analyze room sounds and trigger scientifically proven soothing responses without putting a family's privacy at risk.

What it does

-Local Audio Analysis: It listens to the room's sound patterns entirely within the browser's memory.

-Smart, Safe Responses: When it detects sustained noise (such as crying or stirring), it can play synthesized low-volume responses (white, pink, brown noise, or lullabies) to help soothe the baby tailored to the babies personalized response to such noises - a smart system.

-Absolute Privacy: It never records, saves, or uploads raw audio. Only derived, batched sleep events (like timestamps of noise events) are optionally sent to the backend if the user wants to track long-term sleep progress.

-Safety First: We cap the sound output at a low level to adhere to AAP (American Academy of Pediatrics) guidance (under 50 dB).

How we built it

We split the architecture into a frontend-focused audio engine and a lightweight, optional backend. The Frontend (The Core): Built with HTML, CSS, and JavaScript, relying heavily on the Web Audio API. By keeping the processing in the browser, we eliminated the need for server-side audio processing. To detect noise, we process the audio buffer data locally to calculate the Root Mean Square (RMS) energy of the sound wave. For a given audio buffer array $x$ of length $N$, the energy is calculated as:$$E_{rms} = \sqrt{\frac{1}{N}\sum_{n=0}^{N-1} x[n]^2}$$If $E_{rms}$ exceeds a dynamically calibrated threshold for a specific duration, our DSP (Digital Signal Processing) synthesizer triggers a gentle fade-in of pink or brown noise. The Backend (Optional Analytics): We built a fast, robust backend using FastAPI and Python connected to a PostgreSQL database (orchestrated via Docker Compose). This backend only accepts mathematical feature vectors and timestamps.

Challenges we ran into

-Navigating the Web Audio API: Managing audio contexts, especially dealing with browser auto-play policies (which block audio context from starting without user interaction), required us to build an intentional UI flow to get explicit user opt-in.

-Algorithmic Calibration: Babies make a lot of random noises, which can present potential difficulty. Setting the right threshold so the app doesn't trigger a soothing mechanism for every little sound detected, but rather, responds to actual crying, required fine tuning of our rolling average algorithms.

Accomplishments that we're proud of

-Zero-Data Leakage Architecture: We were able to successfully build a system that provides "smart" features while guaranteeing that raw audio never is leaked from the device.

-The Synthesizer: Using the Web Audio API to generate algorithmic white, pink, and brown noise locally rather than playing looped MP3s. This allows it to be lightweight and high quality in audio.

What we learned

-Deep-dived into the Web Audio API and real-time frontend signal processing.

-Learned how to design a local-first PWA (Progressive Web App) architecture.

-Used a hosting system to make a live environment and learned how to deploy locally on PC

What's next for MindPulse

-On-Device ML: We want to implement a tiny, quantized TensorFlow.js model that runs locally in the browser to distinguish between different types of cries without ever sending out the audio.

-PWA Offline Mode: Enhancing the service workers so the app works flawlessly even if the Wi-Fi completely drops out in the middle of the night.

-Open Source Community: We want to expand our feature vector dataset (strictly opt-in) to train better open-source offline models.