Belugo – Gunfire Multilateration

Belugo homepage
A close up of the heat map available to first responders
An example of what a user's phone would see during a lockdown emergency situation
The dashboard available to law enforcement w/ a heatmap of likely gunshot locations
The phone's recording system debugging

Inspiration

Gun violence has hit close to home for us. Memorial High School, where one of us went, made national headlines over a gun threat. And here in Austin, the shooting on Sixth Street happened just a few blocks from where we relax on our weekends.

In both cases, what stuck with us was how much confusion there was in the first few minutes. Nobody knew exactly where the danger was. 911 calls conflict, sound echoes down hallways, and responders end up clearing rooms one at a time. Every minute spent searching is a minute that matters.

Then we noticed that almost every classroom already has dozens of microphones in it: the phones in students' and teachers' pockets. We wanted to see whether those phones could work together to pinpoint where the shots came from, giving first responders valuable time and helping save the lives of America's next generation.

What it does

Belugo turns the phones already in a building into one acoustic sensor network. It's named after the beluga, a whale that finds its way by listening.

Phones listen. A participant opens a web page, picks where they are (a room, a spot on the floor plan or a pin on a real map) and taps Start listening.
Phones detect a bang on the device. When a phone hears a sudden, sharp impulse, it sends the server just four things: who it is, when it heard the sound, how loud it was and how confident it is. No audio ever leaves the phone to preserve privacy.
The server works out where the sound came from. A sound reaches nearby phones sooner and louder than phones farther away. The server compares every phone's report and finds the location that best explains both, then draws it as a likelihood heatmap over the floor plan or map.
Everyone gets told. Within seconds, the incident dashboard shows the estimated room and every listening phone gets an alert. People can reply with one tap (safe, need help or injured), so responders know where to go first. If the source moves, Belugo draws a trail of each shot.

For judging, anyone can create a private demo space with a QR code, add a few phones (or browser tabs), clap or press Simulate shot, and watch the location appear.

How we built it

Stack: Next.js (App Router) and React with TypeScript, the Web Audio API, Supabase (Postgres + Realtime), Leaflet/OpenStreetMap and Vitest. Our early prototypes were in Python with SciPy and pyroomacoustics.

Detection on the phone. An AudioWorklet runs on the browser's audio thread and keeps a running estimate of the background noise. It flags any onset that rises well above that noise, timed to the exact audio sample. We turn off the browser's echo cancellation, noise suppression and automatic gain control, because auto-gain would flatten exactly the loudness differences we need. A lightweight classifier then checks the shape of the sound: a fast rise followed by a fast decay.

Clock sync and calibration. Timestamps from different phones are only useful if they share a clock. Each phone syncs with the server several times, keeps the sample with the shortest round trip and re-syncs periodically. We also built two calibration steps:

Mic gain: one reference clap, so loudness can be compared across different phone models.
Timing: one clap at a known spot, to measure and remove each phone's constant audio-pipeline delay.

Localization. For each candidate point $(x, y)$ on a grid, we ask how well a sound starting there explains what every phone reported. If phone $i$ is at distance $d_i$, the sound should arrive at

$$t_i = t_0 + \frac{d_i}{c}, \qquad c \approx 343\ \text{m/s}$$

and should be about this loud:

$$L_i = L_0 - 20\log_{10} d_i$$

We don't know when the shot happened ($t_0$) or how loud it was at the source ($L_0$). For each cell we solve for both as weighted averages, then score the cell with a combined chi-squared cost:

$$\chi^2(x, y) = \sum_i \frac{\big(t_i^{\text{obs}} - t_0 - d_i/c\big)^2}{\sigma_{t,i}^2} + \sum_i \frac{\big(L_i^{\text{obs}} - L_0 + 20\log_{10} d_i\big)^2}{\sigma_{L,i}^2}$$

Each phone gets its own uncertainty. For example, its timing error combines the audio pipeline, the clock-sync round trip and how sure we are of where the phone is:

$$\sigma_{t,i}^2 = \sigma_{\text{pipeline}}^2 + \frac{\text{RTT}_i^2}{12} + \left(\frac{\sigma_{\text{pos},i}}{c}\right)^2$$

This lets a well-calibrated phone with a precise position count for more than one placed roughly by GPS. The heatmap on the dashboard is the relative likelihood $e^{-\Delta\chi^2/2}$ of each cell compared with the best one.

Backend. Reports that arrive within $\pm 300$ ms of each other are grouped into a single shot. Supabase Realtime pushes incidents, alerts and status replies to every screen. We also built an admin panel to control every phone remotely, a real-map mode with GPS and a draggable pin, and a simulator. The simulator adds fake clock skew, mic differences and GPS error, then compares the true position with Belugo's estimate.

Challenges we ran into

Timing is unforgiving. Sound travels about 34 cm per millisecond, so a few milliseconds of error moves the estimate by meters. Browsers buffer audio, report latency inconsistently and run each phone on its own clock. Getting timestamps from different phones to line up took several rewrites of our audio timing code, and in the end we needed the timing calibration step.
Phones try to "help." Built-in auto-gain and noise suppression squash loud sounds, which ruined our loudness comparisons until we turned them off.
Indoor GPS is rough. GPS indoors is often off by 15–50 m, which is bigger than a classroom. We added a draggable pin and weighted each phone by how much we trust its position, so one bad fix can't drag the estimate off.
Everyone reports at once. When ten phones hear the same shot, ten requests arrive within milliseconds. Our first version redid the full calculation for each one, and the database writes raced each other. We fixed this with a lease on the incident, so one request handles every report that has come in.
Browsers don't like listening forever. Locked screens and background tabs pause the microphone, so we keep the screen awake and make it obvious when a phone is listening.

Accomplishments that we're proud of

A working end-to-end system: real phones hear a clap, and the dashboard marks the room within a second or two.
Zero bytes of audio leave the phone. Privacy was a design rule from the start, not an afterthought.
Sample-accurate detection and a localizer that understands how uncertain each phone's data is, instead of just picking the loudest phone.
Two map modes (a building floor plan and a real OpenStreetMap view), plus demo spaces so many judges can test at the same time without seeing each other's results.
Tests and a simulator for the math, so we could check the algorithm without firing anything.

What we learned

How much is hidden inside "just record audio in the browser": audio clocks, output latency, buffers and automatic processing all affect when a sound appears to arrive.
Why localization needs at least three phones. With only two, a mirror-image position fits the data just as well.
That modeling uncertainty matters as much as the model itself. Treating every phone as equally reliable gave noticeably worse results than weighting each one by what we actually know about it.
How to coordinate many devices in real time without race conditions.

What's next for Belugo

A real gunshot classifier. Today's detector accepts any sharp impulse (claps, slammed books). We want a trained model that runs on the phone and tells gunfire apart from everyday noise.
Field testing. Measure real accuracy across buildings, floors and phone models, and report average error, not just the best case.
Multiple floors and 3D, since schools aren't flat.
Built for schools: listening that stops automatically after school hours, clear consent and data-retention policies, access controls and human review before any alert goes out.
Working with responders: getting verified locations to school resource officers and 911 dispatch in a format they can act on.

Belugo is a hackathon prototype. It is not a validated gunshot detector, and its estimates should not be used for real emergency decisions.