Loose-Leaf Tunes

Logo
Home Page
Sample Texts (Coda)
Figma Prototype

Meet The Team

We are a group of first year university students who share the same passion for literature and music.

Brandon Jong - product manager from the University of British Columbia

David Kim - backend developer from the University of British Columbia

Nathan Kwon - frontend developer from the University of Waterloo

Michael Leung - backend developer from the University of British Columbia

Inspiration

Have you ever read a book and wondered what sort of music would fill the silence of your room? Maybe even a spirited conversation with a friend prompted you to imagine a melody floating in the air? What if there was a way for us to fill the void? What if we could play music that is tailored to our lives?

Inspired by our love for music during the isolation of the pandemic, we developed Loose-Leaf Tunes for your everyday personalized soundtrack needs. With just a few words and a tap of a button, you can create your own melody! There is no need to sit still in the silence.

Get ready to put on your headphones, and type ‘til your heart’s content!

What it does

Loose-Leaf Tunes is an interactive website that produces a customized soundtrack based on the words the user inputs. This is accomplished through machine learning in two ways: analyzing the mood of the words and composing based on the mood of the words. Together, this helps Loose-Leaf Tunes accomplish its main goal: making reading even more enjoyable by enhancing the experience with music.

How we built it

Frontend

In the planning stages of the project, we used Figma to prototype a UI for our website and used Font Awesome’s CSS Toolkit to design some of the icons. Then, using the components we had designed in Figma, we proceeded to code a website using HTML 5, CSS 3, JQuery and Javascript. Our main logo was created with the help of Canva, and we embedded a Coda document with sample text into our website.

Backend

For the backend part of our project we had two main tasks: analyzing the user inputs and composing/playing machine generated music. We managed to accomplish both tasks using Javascript.

In order to detect the overall mood of the user’s input, we used the Microsoft Azure API to determine the percentage of positive/negative words. These results are then used to dictate the type of music composed by our program.

Then, to compose the music, we created a database of notes based on happy and sad songs we transcribed. Using the principle of Markov Chains, our program tracks the trend of the songs we transcribed, and using probability for each entity, creates a machine generated piece that accompanies the reader’s input.

Challenges we ran into

One of the biggest challenges we faced was utilizing the information we received from the Microsoft Azure API to compose the music. We quickly realised that we had to include more data sets in our composition program since the melodies that were being played back seemed quite pointillistic and abstract.

In addition, we also had a few issues working with such a large amount of code. Thankfully, we used github to collaborate on this project, allowing us to check and edit each other’s work.

Accomplishments that we're proud of

We were particularly proud of the backend work that we had completed on this project. While none of us had experience in machine learning, implementing Markov Chains was a tough but rewarding process. In addition, we believe that with further work, our product would be a viable business idea that we could continue to pursue after Hack the North.

What we learned

In the 36 hours we were given, we had the opportunity to learn numerous programming languages and computer science principles in order to create our website. With varying programming experience between our team members, we each learned something new about programming and designing a viable product.

We also learned a lot more about using HTML 5, CSS 3, and Javascript in order to frontend our website. While most of us had created static websites in the past, we were unfamiliar with making dynamic websites that interacted with the user. Some elements we implemented include animations, buttons, and navigation bars.

With all of our team members working remotely, we found ways to communicate effectively with each other using social media and tele-communication applications. Also, we had to put deadlines on our tasks in order to reach our goals within the time constraints of this challenge.

Last, but certainly not least, we learned how to manage our time effectively and cut out potential features in order to create a minimum viable product. It was definitely challenging to have to put aside some of our great ideas in order to solidify certain key features in our project.

What's next for Loose-Leaf Tunes

One of the biggest things we are hoping to implement in the next iteration of this project is the production of polyphonic compositions, adding onto the single line melodies we were able to create during the 36 hours we were given. In addition, we are also hoping to add additional instruments on top of the piano, such as string, brass, wind, and even percussion instruments. This would produce a symphonic composition that would emulate a concert hall performance based on the words the user inputs.

In order to increase the accuracy of the composition in relation to the mood of the text, we would like to implement sentence-by-sentence analysis rather than analyzing chunks of text. This would allow more mood inflections to be present in the composition, giving a more accurate picture of what the user inputted.

Lastly, we would like to add additional graphics and animations to enhance the overall UI of our project.

Business Viability

Loose-Leaf Tunes taps into two very unique markets: ebooks and film score composition. With the ebook market estimated at 18.13 Billion USD in 2016 (according to Mordor Intelligence), its trajectory only goes up as the market is expected to reach 23.12 billion USD in 2026. With increased user consumption and easy accessibility, we can expect the ebooks market to continue in the same direction.

Meanwhile, the film composition market also continues to grow after the release of movies and TV series only seems to grow exponentially. The demand for composers is high, as the job market is expected to increase by 5.7% in the US (according to careerexplorer.com).

With both of these industries growing so quickly, our team decided to combine both of them and forge our own industry: machine learning composition based on literature. While this may seem like a niche area with very few consumers, we believe that with the proper business and marketing plan, we can soar to new heights.

Reading can become a tedious task if one is not engaged with the book. This happens quite often since books are just a collection of words on a page/screen. With short attention spans, one might decide to watch a movie instead. But what makes great movies so captivating? Other than the amazing actors and actresses, it is the amazing soundtracks and sound effects that help build suspense and enhance the plotline. Our idea is to take what makes movies so awe-striking and apply these principles towards reading. If there were Hans Zimmer soundtracks to classic novels, students would likely complain less about their studies!

However, there is only one Hans Zimmer, and he can only produce so many amazing pieces of music by himself. This is why our focus is on machine learning compositions. We believe that if we train a model by providing enough music samples, we would be able to create some epic compositions. As consumers begin to see the quality of work produced, they would be more intrigued about our product. This is where we consider our business model.

In order to gain revenue from our product, we focus on two main stakeholders: ebook authors and ebook readers. The idea behind our revamped business is that we would have ebooks with machine learning soundtracks for users to listen to as they read along. In order to get more users to try out our product, we would provide a free trial. Then, we would use a subscription model similar to that of Netflix, which would allow readers to access all ebooks/soundtracks, as well as upload their own ebook, with which our program would form a unique soundtrack.

In summary, the reason why Loose-Leaf Tunes would make a good business is because we are tapping into two growing markets, forming our own industry/category, and expecting to reach a large number of consumers based on observations and implementations we made based on the current film and movie industry.