Inspiration

The running joke at HackSheffield 8 was an AI fart-sound generator. We went the other way: something gentle and SFW, a place to collect and rank the best morning bird song. Dawn chorus, but curated.

Two years later we came back and actually built the version we pitched - the dataset, the DSP, the trained model, the deployed app.

What it does

wizzibiz is a Tinder-style feed for real bird recordings.

  • Swipe through five-second clips from xeno-canto. Each card draws the clip's pitch track as a frequency-over-time plot, so you can see the call as well as hear it.
  • After every swipe the CNN's guess is revealed: "Gray-breasted Wood-Wren, 82% confident".
  • Every clip you like trains a logistic regression over the network's 64-dimensional embeddings. Once there are enough swipes it re-ranks the deck towards what you enjoy and reports leave-one-out accuracy rather than a vanity number.
  • "Generate my dawn chorus" blends the pitch contours of the clips you liked and resynthesises the result with Web Audio.
  • Upload a recording and the same network names it server-side: 20 species plus a "not a bird" class.

How we built it

  • Data: 20 species of xeno-canto recordings (tglcourse/5s_birdcall_samples_top20, 3,199 clips) plus 984 ESC-50 environmental sounds as the negative class. 4,183 clips in total, with the WAV bytes inline in parquet, so no external audio tooling.
  • DSP, hand-written: WAV decode, windowed-sinc resampling to 16 kHz, Hann-windowed STFT (512 / hop 160), 64-band HTK mel filterbank, log compression, three one-second windows pooled into a 64 x 32 frequency-time image, per-band standardisation.
  • Model: a small CNN written from scratch in TypeScript - conv 3x3 x10, pool, conv 3x3 x20, pool, dense 64, dense 21 - trained with hand-written backprop and Adam in plain Node. 167,189 parameters.
  • Serving: quantised to int8 (224 kB) and bundled into a Cloudflare Worker. The Worker does the DSP and inference for uploads, keeps swipes in D1, and fits the taste model per session.
  • Front end: vanilla HTML, CSS and JavaScript, with Canvas for the pitch track and the PCA "sound map".

Challenges we ran into

  • Bird song is not speech. The original build ran recordings through Speech-to-Text and searched the transcript for the word "bird". It never worked, and that is exactly why this version starts with a spectrogram.
  • Inductive bias matters. A 2048 to 96 to 64 MLP on the flattened spectrogram plateaued at 33% after 30 epochs. Switching to convolutions reached 41.6% in 12 epochs.
  • Dataset plumbing. The parquet bytes column decodes as UTF-8 by default, and both bird shards numbered their rows from zero, so clip ids collided and some cards shipped the wrong audio.
  • Pitch tracking. Peak-picking per frame locked onto low-frequency rumble and produced a flat 367 Hz line. It needed per-bin spectral whitening and a median filter before the contours looked like birds.
  • Making the edge model agree with training. int8 quantisation with a per-row scale, then verifying that the Worker reproduces the export predictions exactly on a sample of clips.

Accomplishments that we're proud of

  • 41.6% top-1 over 21 classes on held-out data, where chance is 4.8%, with 84% recall on "not a bird".
  • The whole pipeline runs from raw parquet to a deployed Worker in four commands.
  • Inference is sub-millisecond, on a quantised model inside a 182 kB gzipped bundle.
  • The generative step produces something that genuinely sounds like the birds you liked.

What we learned

  • Representation beats model size: the same idea on a transcript is useless, on a log-mel image it works.
  • Evaluate honestly. In-sample accuracy on eight swipes is always 100%.
  • How much of "AI" is really DSP plus a careful data pipeline.

What's next for wizzibiz

  1. More data: 200+ species, and clips tagged by call type so a one-second window is not a coin flip.
  2. Longer context: a wider analysis window with attention pooling over frames, to fix the classes currently stuck at 0%.
  3. Live microphone mode: continuous detection and real-time species ID.
  4. Longer generations: learn taste across sessions and generate whole dawn-chorus sequences instead of a single blended contour.

Built With

Share this project:

Updates

Submission history