Inspiration
We feel music but rarely see how it’s built. DurHack’s themes, Composition and Marshall Wace’s New Worlds, made us ask: what if every song already held a world no one had visited? Orbitune turns a song’s structure into a place you can explore, so its rhythm, energy and shape become things you can see. That also made us think about who else this could be for: people who are deaf or hard of hearing could explore the structure of a song visually, which is a direction we’d love to develop with that community.
What it does
Orbitune turns any song (Deezer search or upload) into a planet you can walk on. Walking around the equator means walking through the song: bass shapes continents, mids raise hills, treble roughens the ground, and the chorus becomes a mountain while the drop becomes a canyon. The atmosphere’s colour comes from the music’s harmony and spectrum. Gemini names the planet and gives it a mood, a biome, a colour palette, plant life and named landmarks tied to the song’s sections. Every planet joins your galaxy, stored in MongoDB. Same song, same terrain, every time.
How we built it
Three.js + Vite: a sculpted icosphere driven by one height function shared by the mesh and the player, beat-reactive effects, and a first-person controller for spherical worlds. Python (librosa): frequency bands, beats, sections and a spectrum descriptor. Gemini API: structured-output world Blueprints (name, landmarks, colour palette), validated before use. MongoDB Atlas: stores each planet’s 20–30 KB recipe, not its geometry. Planets regenerate perfectly in the galaxy. Express + Docker on Render.
Challenges
We struggled to balance the music and ElevenLabs narration. When players approached a landmark, the song often drowned out the narrator, making the story difficult to follow. We didn’t get the music to fade reliably during narration, so we left the feature out of the final demo.
What's Next
Next, we will properly integrate ElevenLabs narration by adding smooth audio fading the music down as you approach a landmark so its story is always clear, alongside on-screen captions. Beyond AI storytelling, we want to expand our current text to pin feature to let users place custom placemarks anywhere on a planet to store personal memories and recorded voice notes. Because every location maps to a specific moment in the song, revisiting a pin replays that personal sentiment at the exact beat or lyric that inspired it, turning each planet into a deeply personal, shareable musical time capsule.