Lulu - Babycam Analyser

Lulu turns any two phones into a private, smart baby monitor that alerts every caregiver and never uploads your child's video.

Comment

Inspiration

Baby monitors force a bad trade. Cheap ones are a stream you have to stare at. Smart ones send your child's video to a company's servers. And parents rarely watch alone: grandparents, babysitters and daycare staff all need to know when something is wrong. We wanted a monitor that is smart, shared and private, and that runs on the spare phone already sitting in a drawer.

What it does

Lulu turns any two phones into a private, smart baby monitor. An old phone becomes the camera, and every caregiver gets alerts on their own phone or a web dashboard.

  • Cry detection that ignores a single cough
  • Safe zones drawn around the crib, with a warning when a child nears the edge and an alert when they leave
  • Position alerts when a child sits up or may have rolled onto their tummy
  • Hazard alerts for scissors, knives, forks or glasses near a child
  • Multi-child tracking with anonymous labels (Child A, Child B)
  • Shared care with invite codes and owner, caregiver and viewer roles
  • Camera health checks (lens covered, too dark, bumped, battery low, offline) so silence is never mistaken for safety
  • Sleep log built nightly from event text
  • Peer-to-peer live view, end-to-end encrypted

Smart suppression keeps alerts trustworthy. An adult picking up the child, leaning into the crib or walking past doesn't trigger a false alarm.

Lulu is an awareness tool, not a medical device.

How we built it

All analysis runs on the camera phone, so video and audio never leave it.

  • Audio: YAMNet on 0.975 s windows classifies crying.
  • Vision: YOLO11n finds people and objects, MediaPipe Pose reads posture, and ByteTrack follows each child. These run through VisionCamera frame worklets and react-native-fast-tflite in an Expo (React Native) app.
  • Live view: WebRTC, peer to peer.
  • Backend: Supabase provides Postgres with row-level security, Realtime (used only for signaling), pg_cron for offline-camera checks, and an Edge Function that sends Expo push notifications.
  • Dashboard: Next.js 16, Tailwind 4 and Recharts, deployed on Railway as an installable web app.
  • Shared core: all decision logic lives in a dependency-free TypeScript package. The app, the dashboard and a replay harness run the same code, so the harness numbers describe what the app really does.

The database stores only short text events such as "Crying detected" or "Child A left the crib", plus heartbeats and settings.

Challenges we ran into

  • Alert fatigue. Early rules fired on coughs, passers-by and parents lifting the baby. We added episode logic, 3-sightings-in-2-seconds confirmation, and caregiver-pickup and adult-leaning suppression.
  • Telling child from adult without face recognition or storing appearance. We settled on a simple rule, and we document its limits.
  • Privacy versus usefulness. Keeping everything on-device while still alerting several people forced us to send events as text and relay video only peer to peer.
  • Proving it works without a library of baby footage. We built synthetic scenarios and a replay harness that reports caught alerts, misses, false alarms per hour and alert delay.
  • Building native ML on Windows without an Android SDK or a Mac.

Accomplishments that we're proud of

  • Video and audio never leave the camera phone.
  • Row-level security tested in a real Postgres engine (PGlite): outsiders can't read anything, caregivers can't promote themselves, viewers can't open live view, and invites are single-use.
  • 30 unit tests and 32 SQL tests, plus a release gate that fails the build if alert quality drops.
  • Across 11 synthetic scenarios (64 minutes), 12 of 12 alerts were caught with 0 false alarms. This checks the alert logic, not model accuracy.
  • Honest limitations are documented in the README instead of hidden.

What we learned

  • Alert quality matters more than detection quantity. A monitor people mute is worse than none.
  • Pure, dependency-free decision logic makes testing and replay easy, and it lets the app and dashboard share behavior.
  • Privacy is easiest to guarantee as architecture: if frames are never encoded or uploaded, there is nothing to leak.

What's next for Lulu - Babycam Analyser

  • Tune on physical phones (pixel formats, delegates, model output layouts) and publish accuracy numbers on recorded clips, not just synthetic ones
  • Better child-versus-adult recognition and position cues for babies under blankets
  • Background operation on iOS
  • Daycare mode with multiple rooms
  • Compatible with real baby cameras
  • Optional TURN relay hosting for reliable live view on any network

Built With

Share this project:

Updates

Submission history