Inspiration
The running joke at HackSheffield 8 was an AI fart-sound generator. We went the other way: something gentle and SFW, a place to collect and rank the best morning bird song. Dawn chorus, but curated.
Two years later we came back and actually built the version we pitched - the dataset, the DSP, the trained model, the deployed app.
What it does
wizzibiz is a Tinder-style feed for real bird recordings.
- Swipe through five-second clips from xeno-canto. Each card draws the clip's pitch track as a frequency-over-time plot, so you can see the call as well as hear it.
- After every swipe the CNN's guess is revealed: "Gray-breasted Wood-Wren, 82% confident".
- Every clip you like trains a logistic regression over the network's 64-dimensional embeddings. Once there are enough swipes it re-ranks the deck towards what you enjoy and reports leave-one-out accuracy rather than a vanity number.
- "Generate my dawn chorus" blends the pitch contours of the clips you liked and resynthesises the result with Web Audio.
- Upload a recording and the same network names it server-side: 20 species plus a "not a bird" class.
How we built it
- Data: 20 species of xeno-canto recordings (tglcourse/5s_birdcall_samples_top20, 3,199 clips) plus 984 ESC-50 environmental sounds as the negative class. 4,183 clips in total, with the WAV bytes inline in parquet, so no external audio tooling.
- DSP, hand-written: WAV decode, windowed-sinc resampling to 16 kHz, Hann-windowed STFT (512 / hop 160), 64-band HTK mel filterbank, log compression, three one-second windows pooled into a 64 x 32 frequency-time image, per-band standardisation.
- Model: a small CNN written from scratch in TypeScript - conv 3x3 x10, pool, conv 3x3 x20, pool, dense 64, dense 21 - trained with hand-written backprop and Adam in plain Node. 167,189 parameters.
- Serving: quantised to int8 (224 kB) and bundled into a Cloudflare Worker. The Worker does the DSP and inference for uploads, keeps swipes in D1, and fits the taste model per session.
- Front end: vanilla HTML, CSS and JavaScript, with Canvas for the pitch track and the PCA "sound map".
Challenges we ran into
- Bird song is not speech. The original build ran recordings through Speech-to-Text and searched the transcript for the word "bird". It never worked, and that is exactly why this version starts with a spectrogram.
- Inductive bias matters. A 2048 to 96 to 64 MLP on the flattened spectrogram plateaued at 33% after 30 epochs. Switching to convolutions reached 41.6% in 12 epochs.
- Dataset plumbing. The parquet
bytescolumn decodes as UTF-8 by default, and both bird shards numbered their rows from zero, so clip ids collided and some cards shipped the wrong audio. - Pitch tracking. Peak-picking per frame locked onto low-frequency rumble and produced a flat 367 Hz line. It needed per-bin spectral whitening and a median filter before the contours looked like birds.
- Making the edge model agree with training. int8 quantisation with a per-row scale, then verifying that the Worker reproduces the export predictions exactly on a sample of clips.
Accomplishments that we're proud of
- 41.6% top-1 over 21 classes on held-out data, where chance is 4.8%, with 84% recall on "not a bird".
- The whole pipeline runs from raw parquet to a deployed Worker in four commands.
- Inference is sub-millisecond, on a quantised model inside a 182 kB gzipped bundle.
- The generative step produces something that genuinely sounds like the birds you liked.
What we learned
- Representation beats model size: the same idea on a transcript is useless, on a log-mel image it works.
- Evaluate honestly. In-sample accuracy on eight swipes is always 100%.
- How much of "AI" is really DSP plus a careful data pipeline.
What's next for wizzibiz
- More data: 200+ species, and clips tagged by call type so a one-second window is not a coin flip.
- Longer context: a wider analysis window with attention pooling over frames, to fix the classes currently stuck at 0%.
- Live microphone mode: continuous detection and real-time species ID.
- Longer generations: learn taste across sessions and generate whole dawn-chorus sequences instead of a single blended contour.
Built With
- cloudflare-workers
- html5
- node.js
- typescript