ClearSight Porch

Inspiration

I was inspired by blind homeowners who are excluded from modern smart doorbells because conventional smartphone notifications require visual video review that is inaccessible when someone is at the front door.

What it does

It analyzes incoming video doorbell streams using multimodal vision, pinpoints whether a visitor is a courier, neighbor, or stranger, and speaks a natural description accompanied by an HRTF 3D spatial audio beacon aligned with their doorstep coordinates.

How we built it

I engineered a Next.js accessibility-first frontend and a Fastify backend connecting Ring Developer Playground video feeds and WebRTC WHEP streaming with Gemini video perception and the Web Audio HRTF panner API.

Challenges we ran into

Meeting the critical sub-2.0-second end-to-end latency constraint from initial webhook receipt to live audio synthesis required aggressive background video pre-warming and optimized WebRTC SDP negotiations.

Accomplishments that we're proud of

I am proud of creating an accessible, screen-free front door experience with a benchmark test suite proving complete end-to-end spatial audio delivery under 1.8 seconds.

What we learned

I learned how binaural Head-Related Transfer Function (HRTF) audio panning gives visually impaired users an intuitive, immediate spatial sense of physical distance that spoken text alone cannot convey.

What's next for ClearSight Porch

Next, I want to bring hardware-level haptic feedback to connected cane devices and integrate directly with Ring's production partner intercom channel for seamless two-way voice conversations.

Built With

• docker
• fastify
• gemini
• nextjs
• react
• ring
• typescript
• webrtc