Inspiration
Once upon a time, in my 20s, I used to play in garage rock bands in my local city, even recording a few albums in studios. Great times in my life that I sometimes miss in my 40s. I've been using Udio for roughly the past year to make full length radio tracks, just for fun, from all music genres. I generated quite a few songs lately, intending to use the best couple of the bunch (in my opinion) for this hackathon. I had never before used AI to generate video, so this was an excellent chance to give that a try. I chose this song to use because it was catchy, pleasant, and truly gave me "real song" vibes.
What it does
Mimics the sound of a summer-movie soundtrack song, perhaps a love drama or romantic comedy.
How we built it
The song was made using Udio. The prompt was: soft-rock love song, male vocals, piano, "a lovely goodbye at sunset". After a satisfying 30 second snippet was generated (in this case, the verse & pre-chorus of this song), it was remixed to make the singing clearer, and to properly frame and tag the lyrics ([verse], [pre-chorus], [chorus], etc), and to make any small edits to words or phrasing. Afterwards, the rest of the song is built out, first generating a [chorus], then building the song out to the end, then finishing the song with its intro. When generating music, my mantra is "repetition, repetition, repetition". To get one 30 second section of song that sounds "right", hundreds must be generated.
The video clips were made using Hailuo (text to video). I ideated out what I wanted for a video to match the feel of the song, and from that developed my "style" prompt: " stop-motion animated paper mache, colorful and vibrant, slow and sweeping motion". Then, for each scene that I wanted, I simply (under that style prompt) described, in text, what I wanted to see. Again, repetition wins the day. Some scenes only took a handful of generations to get what I was looking for. However, most scenes took many many iterations to get it "right". In all, for the video, 32 individual video clips were generated.
The music video, combining all the above, was produced by me using Microsoft Clip Champ. Clip Champ is super easy to use, and fun to use as well, and provided all the tools I needed to assemble and edit the video.
Challenges we ran into
I've never, prior to this hackathon, made AI video (though I have good experience using AI to generate images, music, sounds & speech, 3D models, and Typescripting). Making video with AI was more tedious than other things, sometimes frustrating.
Accomplishments that we're proud of
For being my first attempt at an AI vid, I'm proud of the whole thing.
What we learned
I learned a good bit about editing short videos, and generating AI videos.
What's next for Samuel Dashy - "Say Goodbye to Me"
I could create additional Samuel Dashy songs from his fire-album "Not Ur Boi", should there be any interest in the future.
Built With
- clipchamp
- hailuo
- udio

Log in or sign up for Devpost to join the conversation.