Inspiration
Baby monitors can show parents what is happening, but they still require constant attention. Parents often have to watch hours of footage, even though most of it is uneventful.
A baby's sleep is essential for their health and development, but parents' sleep matters too. We wanted to build a system that could help parents understand what happened without requiring them to continuously monitor a screen.
Questions like:
- Was my baby sleeping peacefully?
- Were there periods of crying or restlessness?
- Did someone need to step in?
- How did the night go overall?
inspired us to build CradleAI.
The long-term vision is to create real-time baby monitor intelligence that understands when something meaningful is happening and notifies parents only when needed. Instead of waking a parent for every small sound, CradleAI could eventually trigger an alert only when a significant event occurs, such as crying-like audio continuing beyond a certain duration.
A secondary motivation was compatibility. Many smart baby-monitor systems are tied to their own hardware ecosystems. We wanted CradleAI to move toward a software-based intelligence layer that could eventually work with different baby monitor sources.
What it does
CradleAI currently works as an MVP where users upload a baby monitor video. The upload flow is not the final real-time product; it is a way to test the analysis and rule engine.
The system analyzes the video and produces:
- Crying likelihood episodes from audio
- Movement episodes from video
- Possible wake-up likelihood
- Caregiver presence events
- Sleep-related statistics
- A timeline of events
- A parent-friendly AI summary
The key idea is that CradleAI separates raw signals from conclusions. Movement does not automatically mean wake-up, and visual cues alone do not prove crying.
How we built it
We built the backend with FastAPI and Python, and the frontend with HTML, CSS, and JavaScript. SQLite is used for local storage. We used OpenAI Codex as our build partner throughout the project.
The analysis pipeline uses multiple tools:
- FFmpeg extracts audio from uploaded videos
- librosa analyzes acoustic features
- OpenCV analyzes motion in video frames
- YOLO detects caregiver/person presence
- GPT-5.6 analyzes selected visual context and generates summaries
- A detection engine merges all signals into one timeline
For crying, CradleAI uses acoustic features like energy, pitch, pitch variation, spectral brightness, rolloff, and MFCC-based timbre change.
The output is a heuristic crying likelihood score, not a medical diagnosis or trained classifier.
For movement, CradleAI uses frame difference, background subtraction, and optical flow. It groups motion into movement episodes and then estimates wake-up likelihood using movement duration, intensity, and nearby crying likelihood.
Challenges we ran into
We ran into multiple engineering issues:
- Fixing CORS issues between frontend and backend
- Installing and using FFmpeg for audio extraction
- Improving processing speed for short demo videos
- Resolving merge conflicts across branches
- Separating movement from actual wake-up reasoning
Accomplishments that we're proud of
We are proud that CradleAI became more than a simple video upload demo. It now has a real multimodal analysis structure.
- Audio-first crying likelihood analysis
- Movement episode detection instead of frame-by-frame noise
- Caregiver context detection
- Temporal reasoning across multiple signals
- An explainable event timeline
- A parent-friendly summary
- Technically honest confidence labels
What's next for CradleAI
We hope to eventually expand CradleAI to include:
- Live camera streaming support
- Trend analysis and comparison across nights
- Automated mobile notifications
- Secure cloud syncing
Our goal is simply to make baby monitors a bit more helpful, saving parents time and reducing the morning guesswork.
Log in or sign up for Devpost to join the conversation.