Murmur: a stethoscope for urban streams

Try it in a minute: open https://murmur-streams.vercel.app, press Try a Coimbra recording, then Listen.

Inspiration

OneAquaHealth asks a hard question: how does the health of an urban stream connect to the health of the people and animals around it? Its citizen app already asks people to describe a stream and say how it made them feel. Two things were missing. First, a biological indicator anyone can collect: birds and amphibians are two of the project's eleven ecosystem-health indicators, and its policy brief calls them natural controls of disease-carrying insects. Second, an answer: the project's own research (D4.2) found that people keep contributing when they can see what their observations mean.

A doctor listens to a heart before running tests. We wanted to give citizens the same first tool for a stream.

What it does

• Listen. Drop a phone voice memo, a field recording, or the 5–10 second video the OneAquaHealth app already collects. Murmur draws the spectrogram and names the birds and frogs it hears, in the city's own language (Portuguese in Coimbra, Dutch in Ghent, Norwegian in Oslo, French in Toulouse). The AI runs entirely in the browser; the recording is never uploaded.
• Confirm by ear. Every suggestion comes with the 3-second clip that triggered it: Did you hear it? Yes / Not sure / No. Windows too noisy to judge are hatched, so silence is never read as absence.
• See what the ear can't tell you. At a OneAquaHealth research site, the result sits beside the lab's pathogen, faecal-bacteria and antibiotic-resistance-gene risk and its fish, invertebrate and diatom quality classes. A recording near site C5 in Coimbra holds five bird species, while the lab found very high microbial risk at C5: a lively stream is not necessarily a safe one.
• One Health reading. Animals (species, habitat indicators, insect eaters), environment (the balance of natural and human sound, how often water masked calls) and people (how much of the time birds were singing, a known wellbeing link).
• Share and verify. People can share the calls they kept. Before a clip leaves the device, human voices are detected and muted. Listeners at home verify clips: three votes and a two-thirds majority agree or reject a call; disputes go to an expert.
• Explore. A map of the 106 OneAquaHealth sites with two layers, what the lab found and what people heard, and a page per site.
• Export. A FHIR R4 bundle shaped by the OneAquaHealth FHIR IG (location-oah, observation-indicators-oah with the IG's own birds and amphibians codes, GBIF taxa, a Provenance separating the model's suggestion from the person's confirmation) and Darwin Core for GBIF.

How we built it

• AI in the browser: BirdNET+ V3.0 (11,560 classes including 647 amphibians) as ONNX on onnxruntime-web in a Web Worker, about 230 ms per 3 s window. The BirdNET geomodel, precomputed for each city and week, keeps species that are plausible there and then; anything else is listed apart, not hidden. Silero VAD screens shared clips for speech.
• Acoustic indices computed with our own FFT: an audibility index (how far calls rise above steady noise) and the NDSI (Kasten et al. 2012).
• App: Next.js 16, React 19, TypeScript, Tailwind 4, MapLibre + OpenFreeMap, Postgres on Neon, zod-validated APIs, deployed on Vercel.
• Data: a snapshot of the OneAquaHealth public API (sites, ecology, health risks, urban context); fourteen public recordings (xeno-canto, iNaturalist, Wikimedia Commons) from the five cities and nearby, including frogs and a short video; GBIF taxon keys; a Python pipeline that regenerates it all.
• Quality: 56 unit and integration tests (including the database on an in-process Postgres), CI on GitHub Actions, FHIR validated on the public HAPI R4 server (no structural errors), axe accessibility checks with no violations at desktop and phone widths.

Evidence (numbers a judge can check)

• Noise robustness: 167 clean calls of 14 stream-side species mixed with real flowing-water recordings (1,002 mixtures). With water as loud as the birds, Murmur still found 84% of calls; with water alone it named a plausible species in 1.1% of windows.
• The browser reproduces the reference run: 63 of 72 species-recording pairs agree across the ten public recordings.
• In OneAquaHealth's own data, 90 of 96 lab samples were taken after three dry days. Recordings can be made any day, including after rain.

Challenges

• Running an 11,560-class model inside a browser tab fast enough to feel instant, and keeping it identical to the reference run.
• Being honest about silence: rushing water hides birdsong, so we measured how much, and show it.
• Privacy by design: voices had to be removed on the device before any sharing.
• Fitting citizen observations into a health-oriented FHIR guide that was built for lab data.

Accomplishments

• A working, deployed product, not a mock-up, built on the project's real sites, lab data and FHIR model.
• A stream assessment anyone can do with their ears, even from home.

What we learned

Sound is a voucher: a recording lets anyone check an identification later, which makes citizen data verifiable. And the most persuasive One Health message came from putting two datasets side by side, not from a score.

What's next

• Add Murmur as an optional step in the OneAquaHealth app, reading the audio of the video it already collects.
• Propose citizen performer, verification status and soundscape codes for the OneAquaHealth FHIR IG.
• Night recordings for amphibians, and seasonal "dawn chorus" listening events across the five cities.
• Calibrate against OneAquaHealth's expert bird and amphibian surveys.

Built With

• birdnet
• darwin-core
• gbif
• hl7-fhir
• maplibre
• neon
• nextjs
• onnxruntime-web
• openfreemap
• pglite
• playwright
• postgresql
• python
• react
• silero-vad
• tailwindcss
• typescript
• vercel
• vitest
• web-audio-api
• webassembly
• zod