Q - Play Your Story

Feedback on a DJ set takes an expert an hour of listening, so it rarely happens. Q does the listening: every transition scored, every fault timestamped, coaching report in minutes.

Comment

Inspiration

I play a one-hour DJ set every week. Thirty-something episodes so far. This is Let it Beat. Listen in background: link

The only way to actually get better is for someone experienced to sit down, listen to the entire hour, and tell you — with timestamps — where the transition landed late, where two basslines fought each other, where the energy arc lost the room.

That is an hour of expert attention per set. It doesn't scale, so it doesn't happen. Not for working DJs, and not for students in DJ schools who get one instructor between forty of them.

Every other craft has a feedback loop. Mixing has a bottleneck made of real-time listening.

I wanted to remove it, so I built an agent that does the listening.

What it does

You upload a set recording and, optionally, your Rekordbox library export. After that there is no human in the loop.

Q analyses the signal. BPM curve across the full hour, spectral flux, sub-bass energy in the 30–100 Hz band, onset and beat grid, energy envelope, and the section boundaries where one track gives way to the next.

It identifies what was played. Every detected section goes out for acoustic fingerprint identification, enriched with harmonic metadata, and cached so a track you have played before is never re-identified.

It scores the mixing, not the music. Each transition gets a mix type — EQ blend, filter fade — a beat-sync verdict, a harmonic-compatibility judgement on the Camelot wheel, and a bass-clash penalty derived from measured low-end overlap rather than a rule of thumb.

It scores the set against you specifically. Supply your Rekordbox export and Q computes a DJ Identity match: how far this set sits from the BPM range, key palette and tag vocabulary you actually play. The feedback is calibrated to your style, not to a generic ideal.

It runs a multi-agent critique. A Gemini 3.5 evaluation pass reads the measured evidence from several critic perspectives, flags anomalies, and generates specific remediation drills.

It produces the artifact. An interactive dashboard and a downloadable PDF coaching report with timestamped findings and per-track practice drills — the document an instructor would hand to a student.

It answers questions about it. A conversational agent grounded strictly in that set's measured data, so you can ask why did transition 7 score low and get an answer tied to the numbers.

Every set is saved to a library, and a progress endpoint reads your past sets to report whether you are actually improving.

How I built it

A single upload triggers a five-stage autonomous pipeline: ingest → analyse → evaluate → persist → deliver.

An orchestrator fans out to three agents in parallel:

  1. DSP analyzer — librosa and scipy over the raw waveform.
  2. Audio intelligence — acoustic fingerprint identification, harmonic enrichment, and a Firestore-backed track cache checked first.
  3. Rekordbox parser — the DJ's own library as ground truth for style, key and genre.

Their merged output goes to a Gemini 3.5 evaluation pass through the Google GenAI SDK, which produces the critique, the anomaly register and the remediation drills as structured Pydantic schemas. The PDF is then composed and the dashboard populates.

The split between agents is not decorative. Each has a genuinely different failure mode and a different external dependency: the DSP analyzer fails on corrupt audio, identification fails on catalogue gaps and rate limits, the parser fails on malformed XML. Isolating them means one failure degrades the report instead of taking the pipeline down.

Stack

  • AI — Gemini 3.5 via the Google GenAI SDK
  • Cloud — Cloud Run (backend and frontend, me-west1), Cloud Firestore, Cloud Storage
  • Backend — Python 3.11, FastAPI, Pydantic, librosa, scipy, NumPy, soundfile, ReportLab
  • Frontend — Next.js, React, TypeScript, Tailwind, Recharts
  • Built in — Google Antigravity

Data sources

  • Set recordings supplied by the user — WAV, MP3, AIFF, FLAC, typically 60-minute master recordings
  • Rekordbox XML library exports — BPM, Camelot key, genre, ratings, play counts, personal tag vocabulary
  • AudD API for acoustic fingerprint identification
  • Spotify Web API for key and mode enrichment, mapped to Camelot notation
  • Per-genre calibration profiles, because a progressive house blend and a peak-time techno cut have completely different boundary signatures

Challenges I ran into

A four-times hop-length mismatch was corrupting every number the system produced.

Tempo estimation ran its onset envelope at hop_length=512 while the beat tracker downstream assumed a different resolution. The BPM curve wobbled ±5–10 across the whole hour. It looked like noisy audio. Every derived judgement inherited the error — beat sync, transition timing, drift, stability — all computed from a curve that was wrong in a way that looked entirely plausible. Dropping to 128 everywhere fixed it, at four times the compute cost.

The lesson was not "check your hop lengths." It was that a signal-processing bug and a bad model output look identical from the outside: both produce a confident, wrong number, and neither raises an exception.

librosa reports tempo an octave out, convincingly.

Autocorrelation locks onto half or double the true tempo, so a 124 BPM house track comes back as 63.5 or 248. A naive median inherits it, and everything anchored to that median inherits it too.

The fix took three layers. Iterative octave correction folding outliers into a window around the running median \( [0.6\,\tilde{m},\; 1.4\,\tilde{m}] \), then a trimmed median over the inner 80% of the curve to exclude fade-in and fade-out, then a tight clamp. Order matters: octave correction has to run before the trimmed median, or the median it corrects against is itself corrupted.

Narrowing the search window from 40–220 BPM to 100–150 removed an entire class of error for free. Domain knowledge as a search constraint is underrated.

A one-hour set does not fit in a Firestore document.

The BPM curve, energy envelope and spectral arrays blow past the 1 MB limit. Firestore now holds structured state — status, scores, tracks, transitions, the library, the track cache. The dense arrays are gzipped into Cloud Storage and fetched only when a chart needs them.

The constraint produced a better design than I would have chosen freely: the dashboard loads its summary instantly and pulls the heavy arrays only on demand.

Acoustic fingerprinting has a hard ceiling on underground music.

On released tracks it is near-perfect. On white labels and unreleased edits it drops to roughly half, and there is no clever fix — an edit a producer sent a friend over WeTransfer is in no catalogue anywhere.

So I did not build one. The pipeline continues on signal analysis alone and the report states plainly what it could not identify.

That was a real decision and I nearly went the other way. It is tempting to let the model fill the gap — it would produce a plausible title and the tracklist would look complete. But a coaching tool that invents a track name is worse than one that admits a gap, because the moment you catch it inventing once you stop trusting the numbers too. And the numbers are the whole product.

Accomplishments that I'm proud of

The agent does the work rather than describing it. Between the upload and the PDF there is measured signal, tool calls, state transitions and a generated artifact — no point at which a human decides anything.

And it does not make things up, structurally rather than by hope:

  • Every figure in the report traces to a measurement.
  • The conversational agent is constrained to that set's data and instructed never to state a number that is not in it.
  • The orchestrator validates its model string against a Gemini 3.5+ allowlist at startup and refuses to run otherwise.
  • Mock mode cannot reach production — the container refuses to start if the mock flag is set while the Cloud Run service identifier is present. Fake data cannot leak into a deployment by accident, because the process exits first.

What I learned

The hard part of an agentic system is not the agent.

It is the fault tolerance around it: what happens when a third-party API returns an empty array instead of an error because it ran out of credits, when the audio is clipped, when the model returns something that does not fit the schema. Orchestration was the fast part. Surviving contact with a real 78 MB AIFF file was the slow part.

I also learned to trace every value the interface displays back to the line that computed it. That habit is what eventually found the hop-length bug — not listening harder, but reading the call chain.

What's next for Q

A local calibration harness: ground-truth fixtures scored against the engine's output, so DSP parameters get tuned against measured accuracy instead of listening once and guessing.

And faster, cheaper identification for tracks the system has already seen.

Thanks

To the team at AudD, who provided API credits for this project. Identifying tracks inside a continuous mix — no metadata, no boundaries, one long file — is a harder problem than identifying a single song, and being able to iterate on it without watching a meter made a real difference.

Pre-existing code disclosure

Q was built during the submission period, 3–31 August 2026, in Google Antigravity.

It uses standard open-source libraries and frameworks, each under its own licence: FastAPI, Pydantic, uvicorn, librosa, scipy, NumPy, soundfile, ReportLab, matplotlib, Next.js, React, Tailwind CSS and Recharts. Third-party services are used under their published terms: the Google Gemini API, Google Cloud Platform, the AudD music recognition API and the Spotify Web API.

For completeness: before the submission period I had written a small personal command-line tool for parsing my own Rekordbox exports and computing Camelot-wheel harmonic distances. Q is new code written for this hackathon, and that earlier personal work is disclosed here.

No part of this project was developed with financial or preferential support from Google or Devpost.

Built With

Share this project:

Updates

Submission history