Lumi Chords

Inspiration

Inspiration in this idea

I was inspired by the Guitar as an instrument. It is one of the most played instruments in the world. When I practiced playing the guitar myself, I encountered various problems such as forgetting chords, can't change chords in time then when I go to find solutions whether it's in the app or the device. It doesn't have a device or app that can fix the problems I'm having. That’s why me and my friend came up with this idea to solve our pain point. Moreover, I want people to access it more easily and want to create an enjoyable experience for newbies trying to learn it.

Inspiration in this AI

This AI specifically is inspired by the online classes provided by a number of providers. It could be seen that those classes aren’t that effective if the sounds played by the learners can be analyzed to find the mistakes in the playing process. This could be an effective learning tool for this instrument.

What it does

*What can this AI do *

Basically, this AI can distinguish the sound of a song playing and tell what chord chances it will be. This AI can distinguish up to 12 chords, C, C♯, D, D♯, E , F, F♯, G, G♯, A, A♯, B.

The purpose

This AI can be used in analyzing and creating online lessons or analyzing the mistakes in the song played by the user. Our main project is a software that provides lessons and trains our clients. The AI could help in determining the chords from the songs and speed up the lesson creation process.

Step-by-step

Upload the audio file into librosa in type “.wav”
After that change the time domain into “chroma” by using Short Time Fourier Transform (STFT), then change the numpy array to only mean value. The reason why we have to change it into mean value is to make the size of the chroma feature smaller. If it’s a default value we need to record the sound at exactly 5 seconds which is too hard for people to do it. Then arrange to insert it in a data frame and convert into .csv and use .csv to train AI.
After the data preprocessing we will use the processed data to train the AI in 20 epochs which concludes the training part. Then In the functionality part, after we input a .wav file it goes through 2 layers of dense neural network with 32 & 7 nodes. The first layer uses relu activator, and the second layer uses sigmoid to get a value of 0-1 to convert it into percentage. The AI then breaks the file into 32 then 7 nodes to get the seven chords then print the one with the highest probability according to the sigmoid activator.

How we built it

What we’ve used to create this AI

Firstly, we don't have our own dataset, so in order to start we search for a dataset in each chord. We have decided to use AI to determine only 7 chords but the data set has 12 chords. Then we used librosa to convert the files to frequency domain by using Short time Fournier's Transform(STFT). After that we use numpy arrays to convert all that datas into an average value for easier usage. Then put all those datas into a data frame, convert the files into .csv. and use the files to train the AI.

Challenges we ran into

Challenges we faced

Due to not having much background on coding and computer programming we have been struggling with this project quite a lot. Plus, we don’t have our own dataset, and the data set we found needed to be converted to usable types of data before being used with machine learning. Moreover, with the sound files there is a concern of noises which have to be removed.

How we overcome the challenges

We have enlisted our friend’s help from multiple faculties including engineering and computer science to help guide us in this project. As for the data collection, we have to go to a musical database to find it.

Accomplishments that we're proud of

We’re proud of our accomplishment from creating this project and gaining increased knowledge in an unfamiliar field.

What we learned

We’ve learned alot about datas, musical theory, artificial intelligence and neural networks in order to do this project

What's next for Lumichords

We will try to do a real time analysis next, since that's our current limitation. If a real time analysis could be made more opportunities follow including voice recognition and more.