Inspiration
Me and my teammate both love music ! Listening to it on drives, hikes, workouts, on the commute! The only issue is we can't create music :(. In the age of LLM's , when even a non coder can vibe code their way to a functioning app then why can't we be able to make music. To help ourselves and millions more like us vibe code music we made Jamflow.
Our Solution
We solved this problem by creating Jamflow. We built a tool where a user can enter a prompt. Have Gemini 2.5 flash go through it and then actually generate working code that you can hit play on and listen to in the chat itself!! thats not it you can even edit the code and hit play to immediately see how your unique melody is whipping up.
How we built our Project :
We built our project by fine - tuning gemini 2.5 flash on a custom dataset that was curated by scraping extensive music documents and guides. The db was then converted to a vector db to be able to implement RAG. When a user now enters the prompt, we convert that prompt to vector embeddings, Query the vectorDB and then pass this as context to the llm via api calls. This ensures we can always generate relevant strudell code and output exactly what the user expects.
What we learnt
To begin with, we can confidently say that we have a pretty solid understanding of Strudell code. We even learnt a lot of the ui ux as this was the first time either of us designed the front end of an application in a hackathon and also the first time we ever designed a chatbot .
The challenges Faced
To begin with, there was no curated dataset available. It was practically impossible to fine tune the llm to generate accurate strudell code as we did not have a pre curated dataset of strudell docs, musical understanding research papers. Another challenge faced was we had to figure out how the user query would be converted to vector embeddings to be able to reduce latency and improve speed
Log in or sign up for Devpost to join the conversation.