Jam Frog

Thank you to the team behind Hacklanta and Progsu for making this possible!

Jam Frog is an AI bandmate for Ableton Live: play a phrase, and a VERY CUTE frog answers with music you can build on. It listens, suggests your next move, and jams in time while you keep your hands on the instrument.

Inspiration

We built Jam Frog around a familiar creative roadblock: you find a chord progression you like, then get stuck deciding what comes next. We wanted a collaborator that could listen, offer ideas, and take a musical turn while you stay in your creative flow. The frog gives that collaborator a playful personality.

What it does

Jam Frog listens to MIDI chords and suggests possible next moves. In Answer mode, it responds to a two-, four-, or eight-bar phrase on the next downbeat. In Jam mode, it keeps playing progressions you can improvise over, previews the next phrase, and follows changes in your key. Its responses can mirror your rhythm and dynamics, while chord voicings stay near the register you played.

Six genre presets steer the predictions and switch instruments in Ableton. A weirdness knob moves between familiar progressions and more adventurous choices, including borrowed chords and secondary dominants, while keeping the music connected to the key. Guitar genres can trigger Ample Guitar strums, and players can display chords as names or Roman numerals. After a jam, a report highlights key changes, substitutions, cadences, and recognizable progressions.

How we built it

We built the desktop overlay with Electron, TypeScript, and Canvas, connected to Ableton through two custom Max for Live devices: Frog Ear receives the player's notes, and Frog Voice plays the frog's responses. Notes, tempo, and bar timing travel over localhost using OSC.

Alex handled Ableton integration, Katie developed the GRU model, and Kevin built the visual interface, genre animations, and showcase site. Shared interfaces, browser demos, and tests let us work in parallel even though only one teammate had Ableton available.

For chord prediction, we trained an n-gram baseline and a genre-conditioned GRU on Chordonomicon, representing chord movements relative to the previous root so patterns could transfer between keys. We exported the trained weights and implemented local GRU inference in TypeScript, alongside an ONNX export.

On the same 20,000-section test sample, the GRU reached 61.5% top-1 and 83.3% top-3 accuracy, compared with 56.2% and 79.6% for the n-gram. We built a blind A/B listening test because predicting a recorded song's next chord and producing a continuation someone enjoys are different goals.

Giving the frog personality

The frog sits on a pixel-art amp whose controls occupy the head and whose cabinet displays the chord timeline. Genre-specific outfits, instruments, and amp designs make each style recognizable. Tempo-aware performances, calmer idle motion, occasional gestures, and a three-frame tongue animation give it character.

We shared sprite assets between the app and a responsive showcase website, which also includes an interactive signal-path walkthrough and a browser-rendered illustrated demo.

Challenges we faced

Our hardest challenges were making the system behave like a musician: grouping slightly staggered MIDI notes into chords, recognizing inversions and power chords, landing answers on bar boundaries, handling interruptions, and avoiding stuck notes when instruments change.

We also had to balance surprise with coherence and merge simultaneous changes to the models, interface, artwork, and playback engine.

What we learned

A useful musical assistant needs more than a good prediction model. Timing, voicing, responsiveness, and clear controls determine whether an idea feels playable. We also learned how much shared contracts and mock environments help a small team build together.