Harmonix

Harmonix Website Homepage

Inspiration

Harmonix was inspired by the growing popularity of audio content and the desire to make online information more digestible to those who prefer listening over reading, those who want to learn on the go, and those who have a hard time recalling facts. We wanted to create a tool that could seamlessly transform web pages into personalized soundscapes, enhancing the browsing experience by adding an auditory dimension.

What it does

Harmonix is a Chrome extension that generates personalized audio content based on the websites you visit. Whether you're reading a blog, exploring an article, or browsing an online store, Harmonix creates custom music, energetic EDM tracks, or podcast-style summaries tailored to the content. Users can save these audio tracks directly to their YouTube accounts for easy playback anytime, enriching their browsing experience with sounds uniquely suited to each page. Additionally, users can deposit Solana to gain ownership of the audio.

How we built it

We developed this comprehensive solution, combining a Chrome extension with a website built using Next.js, React, and Vite for a seamless user experience. The backend infrastructure leverages Supabase for database management, while audio generation is powered by Twelve Labs' models for speech-to-text podcast and EDM creation, and Suno's Chirp model for song generation. To integrate these components effectively, we utilized LlamaIndex, with GPT-4 handling much of the heavy lifting in terms of content processing and generation. The majority of our codebase is written in Python, ensuring robust functionality and scalability across the platform. This tech stack allows us to offer users a unique audio NFT purchasing experience, transforming web pages into personalized soundscapes and fostering a new way of engaging with online content.

Challenges we ran into

Throughout our development journey, we've undergone numerous iterations to identify the optimal LLM text-to-speech model and music generation system, leveraging the cutting-edge technologies provided by our sponsors. This process has been incredibly educational, deepening our understanding of NFTs, Web3 technologies, and the Solana blockchain ecosystem. The challenge of seamlessly integrating these diverse components into a cohesive platform was significant, but it pushed us to innovate and problem-solve creatively. We're particularly proud of how we've managed to harness the power of Twelve Labs' audio models and Suno's Chirp model to create a unique audio experience, while utilizing Supabase for robust database management. The support from our sponsors has been instrumental in overcoming technical hurdles and bringing our vision to life, resulting in a product that we believe truly transforms the way users interact with online content through personalized audio NFTs.

Accomplishments that we're proud of

Seamless generation of contextual audio, integration of cutting-edge tools, prompt engineering for EDM, Improving accessibility, innovative NFT mining

What we learned

We learned that early feedback was important for refining EDM prompt generation and improving quality. LlamaIndex and OpenAI are powerful tools when used together as they generate meaningful outputs from scraped, unstructured data. Additionally, focusing on real-time playback and accessibility was key to enhancing user experience.

What's next for Harmonix

We're aiming to incorporate more gamification features such as fill-in-the-blanks to test user knowledge on the content and deploy a freemium pricing strategy to earn revenue through consistent users.