ThinkInMusic

Landing page of web app.
Generation in the backend.
Emotions being detected.
Emotions Detection, and Music Generation complete.

Inspiration

"Meandering through the cobblestone streets of Florence, Italy, a golden sunset painting the Renaissance-era architecture, the air was filled with the aromatic fusion of freshly brewed coffee and baking pizza, the timeless echo of church bells punctuating an unforgettable exploration of Tuscan art and history." What emotions ran through your mind when you lived this moment? What kind of music would those translate to?

Builders build. When you really want a tune that only you can create, you shouldn't have to learn all the instruments before you can create music. What if you could do it with a few words, and a few clicks?

What it does

ThinkInMusic asks you to describe a memorable scene. It then extracts emotions out of that scene, and generates music to match the scene.

How we built it

We extract emotional signals from the end user's prompt, using the text APIs provided by Hume. Using the scene description, and emotions we generate a template that is then used by OpenAI's chatGPT to generate another prompt for a music generating Diffusion model (AudioLDM). Finally, we use AudioLDM to generate the perfect melody to match your mood.

Challenges we ran into

Various integration points between multiple large models. GPU instances were scarce. Front end integration on a hosted notebook required some tradeoffs.

Accomplishments that we're proud of

Combining multiple aspects of the human experience into an application that is both technologically advanced, and yet so human.

What we learned

We as a community are in the nascent stages of a massive democratization of the creative potential that AI is going to unleash. With every new model, we are now able to reduce the barrier of entry into AI and its applications.