Syntheo

Our logo

Inspiration

At Syntheo, we understand the power of music and its ability to evoke emotions and tell stories. However, we also recognize that creating a truly personalized music experience can be time-consuming and challenging for many individuals. Our inspiration for Syntheo stems from a desire to bridge this gap and make the process of creating personalized music videos accessible to everyone. Research has shown that it can take years of dedication and practice to develop the skills necessary to compose, produce, and perform original music. This often leaves people feeling disconnected from their own musical journey, unable to express their unique stories and emotions through song. We saw an opportunity to harness the power of artificial intelligence to streamline this process and empower individuals to create their own personalized music videos with ease. By engaging in a simple conversation with our chatbot, users can share their thoughts, feelings, and experiences, which our AI technology then translates into a one-of-a-kind music video. Our mission is to make the joy of creating personalized music videos accessible to all, fostering a deeper connection between people and the music that moves them.

What it does

Syntheo is an innovative platform that simplifies the process of creating personalized music videos. By engaging in a conversation with our friendly chatbot, users can share their thoughts, emotions, and experiences. Our advanced AI technology then analyzes this input and generates a unique music video that reflects the user's story and feelings. The chatbot interaction is designed to be intuitive and engaging, allowing users to express themselves freely without the need for musical expertise or video editing skills. Through this conversation, Syntheo's AI gains a deep understanding of the user's perspective and crafts a music video that authentically represents their journey. The resulting music video is a powerful and emotionally resonant creation that combines visuals, music, and lyrics to tell the user's story in a compelling way. Each video is entirely unique and tailored to the individual's experiences and emotions shared during the conversation. Syntheo makes the process of creating personalized music videos accessible to everyone, regardless of their musical background or technical abilities. By leveraging the power of AI, we empower users to express themselves through music and share their stories with the world in a way that is both meaningful and effortless.

How we built it

Syntheo is an innovative application that simplifies the process of creating personalized music videos. The frontend, built using React, provides a user-friendly interface where users can initiate the creation of their unique music video. Instead of a traditional backend setup, Syntheo integrates a Hugging Face Space instance, which hosts a Streamlit app serving as the interface for the Gemini chatbot. Users engage in a conversation with the Gemini chatbot through the Streamlit app embedded in an iframe on the website. During this interaction, users provide key information such as the desired title, lyrics, and genre for their personalized music video. The collected information is then sent to a Vercel domain, where the music generation process takes place. To generate the music, Syntheo utilizes the Suno API, a powerful tool that converts text into high-quality music. The Suno API takes the user's input and creates a unique musical composition based on the provided parameters. The Vercel domain, managed using Databricks, a fast API, ensures efficient communication and data management throughout the music generation process. The user's input from the chatbot is also fed into a fine-tuned cascaded diffusion model called 12VGen-XL. This advanced AI model processes the user's preferences and generates relevant information and parameters for the music and video generation. The cascaded diffusion model plays a crucial role in creating a cohesive and personalized experience by aligning the music and visuals with the user's input. Finally, based on the user's input and the processing done by the 12VGen-XL model, Syntheo generates two files: one containing the personalized music and another containing the corresponding video. These files represent the final output of the Syntheo application, providing users with a unique and personalized music video that reflects their preferences and creative vision. By leveraging cutting-edge technologies such as React, Hugging Face Spaces, the Suno API, and the 12VGen-XL cascaded diffusion model, Syntheo offers a seamless and intuitive way for users to create their own personalized music videos, making the creative process accessible to everyone.

Challenges we ran into

One of the biggest challenges we ran into was finding a way to create the music from the user’s conversation with Gemini. Resources for this task are scarce, especially free resources that allow you to download the output. Fortunately, we found Suno, which is a relatively new site that turns a prompt into music. The second challenge we ran into was finding a way to store the Vercel domains, which we were able to do using Databricks. The last challenge we ran into was finding a free and reliable model to create the videos using the prompt. To conquer this challenge, we read research papers to learn about cascading diffusion models, which we used for this specific task.

Accomplishments that we're proud of

Syntheo's accomplishments include successfully integrating multiple cutting-edge technologies, such as React, Hugging Face Spaces, the Suno API, and the 12VGen-XL cascaded diffusion model, to create a seamless and intuitive user experience. We are proud of developing an AI-powered platform that empowers users to create personalized music videos through a simple conversation with a chatbot, making the creative process accessible to everyone. Additionally, we overcame challenges related to music generation, data storage, and video creation by finding innovative solutions such as the Suno API, Databricks, and cascading diffusion models.

What we learned

Throughout the development process, we learned the importance of researching and exploring new technologies to overcome challenges and enhance the user experience. We also recognized the power of integrating various AI models and APIs to create a cohesive and personalized product, as well as the significance of making creative tools accessible to a wider audience, regardless of their technical skills or musical background.

What's next for Syntheo

Looking ahead, Syntheo plans to continuously improve the AI models and algorithms to generate even more sophisticated and personalized music videos. We aim to expand the platform's features and capabilities, such as offering more customization options for users and incorporating additional creative elements. Furthermore, we hope to collaborate with artists, musicians, and content creators to showcase the potential of AI-generated personalized music videos. Exploring partnerships with music streaming platforms and social media networks to integrate Syntheo's technology and reach a broader audience is also on our roadmap. Finally, we will gather user feedback and iterate on the platform to ensure it remains user-friendly, engaging, and aligned with users' needs and preferences.