EchoSense

Inspiration

Most gesture-sensing tech relies on a camera or a dedicated chip — think Google's Soli radar. Cameras raise privacy concerns and don't work in the dark; specialized sensors don't exist on the hardware most people already own. We wanted to see if we could get real gesture sensing out of parts every laptop already has: a speaker and a microphone. Sonar and Doppler radar are old, well-understood physics — we just wanted to see if sound waves at frequencies humans can't hear could do the same job.

What it does

EchoSense turns a laptop's built-in mic and speaker into a contactless gesture sensor. It plays a continuous tone just above the range of human hearing (~19kHz) out of the speaker. When a hand moves near the device, the reflected sound comes back very slightly shifted in frequency — the same Doppler effect that makes an ambulance siren change pitch as it passes you. We run a real-time FFT on the incoming microphone audio, isolate that tiny frequency shift, and classify it into gestures: approach, retreat, and wave. The live spectrogram and frequency-shift graph let you actually watch the signal move as you gesture, so it's not a black box — you can see the physics happening.

How we built it

Web Audio API to generate the ultrasonic tone via an OscillatorNode and to capture raw mic input through an AnalyserNode, with echo cancellation, noise suppression, and auto gain control all disabled since they actively fight the signal we're trying to detect.
A large FFT (32,768 points) rather than a default small one, because Doppler shifts from hand-speed motion are only tens of Hz — a small FFT's frequency resolution is too coarse to see them at all.
A simple heuristic classifier that watches the frequency offset from the carrier tone over a rolling window: sustained positive shift reads as approach (blue shift), sustained negative as retreat (red shift), and rapid sign-flipping as a wave. Debounced and cooldown-gated to avoid flicker.
A live spectrogram and shift-over-time chart drawn straight to canvas every animation frame, so the signal processing is visible in real time during the demo, not just the output.

Challenges we ran into

Finding the actual usable frequency range was hardware-dependent — laptop speakers and mics roll off steeply above certain frequencies, so the carrier frequency needed to be tunable rather than hardcoded.
Getting a frequency resolution fine enough to see a signal that's only tens of Hz wide, without the FFT latency becoming so large it made the gesture detection feel laggy.
Filtering out false positives from room echo and other stray noise near the ultrasonic range — the sensitivity threshold needed to be exposed as a live-adjustable control rather than a fixed constant.

Accomplishments that we're proud of

Getting real signal processing — FFTs, windowing, Doppler shift extraction — working entirely with commodity audio hardware and no external libraries beyond the Web Audio API, with a live visual demo where you can wave at the screen and watch it register in real time.

What we learned

How much of "sensing" is really about signal-to-noise ratio and hardware limitations rather than the algorithm itself — the classification logic is simple, but getting a clean enough signal to classify was the actual hard problem.

What's next for EchoSense

A trained lightweight classifier instead of hand-tuned thresholds, so it can recognize a wider gesture vocabulary.
A Flutter/mobile port so it works as a proximity sensor for phones and tablets, not just laptops.
Multi-mic setups to recover directional information (left/right swipes), which a single mic can't distinguish on its own.