Sound Synth

One device, countless instruments, endless fun!

Comment

Inspiration

Both the Sound Synth members are music lovers and it made perfect sense to follow the Arts path for this hackathon. Between the two of us, we play about 6 instruments (piano, flute, clarinet, guitar, trumpet, and saxophone). We wanted to build something fun and exciting which wrapped traditional/classical music with tech and featured unlimited extensions.

What it does

Sound Synth is an all in one instrument intended for music lovers of all ages. From beginners to skilled musicians who want to have fun and create some sick tunes! It can be used on the go, or at home with ANY computer.

Unlike striking, strumming, or blowing, the Synth works through an ultrasonic sensor to determine the note based on proximity to the device. The device features two modes:

  • Portable mode: the on board buzzer produces the sound
  • Instrument mode: web application with different instruments sound and sheet music creation

How we built it

Hardware

The hardware for this device utilizes an Arduino which processes the ultrasonic sensor reading between specific interval. The value of the sensor determines which note should be played and it's duration. Additionally, there is an LCD display along with a button to toggle between the portable and the instrument modes. The portable mode uses the sensor readings to play the respective notes on the buzzer

Web Application

In instrument mode, the Arduino sends the ultrasonic sensor readings over a serial port to a computer. To start playing, the user can navigate to the web application build on HTML, CSS, and JS to choose their instruments and hit play. This process calls into our python-flask backend application to spawn a thread and read the data from the serial port to play the right notes. When ending your Synth session, the user hit Stop which sends a GET request to the flask app and terminates the thread. Additionally, the entire session is recorded and transcribed into sheet music through LaTeX.

Challenges we ran into

  • Sending and reading data between our backend and Arduino
  • Working with only one Arduino between both of us
  • Transcribing the sheet music
  • Thread manipulation to allow the main server application unblocked

Accomplishments that we're proud of

  • Having a finished product
  • Learning how to communicate between a flask app and Arduino
  • Construct a real-time low latency device with possible applications in IoT

What we learned

In addition to learning new framework such as Flask, we also learned how to work with sound files in Python and making an Arduino act differently based on whether it is connected to USB port or not. Additionally, we also researched about synthesizing sounds with sine functions to produce musical notes resembling specific instruments.

What's next for Sound Synth

Short-Term

  • Adding more instruments
  • Refining user interface
  • Reducing latency for sheet music creation
  • Resolving unnoticed bugs and refactoring inefficient code

Long-Term

  • Incorporating a mixer to layer several sessions for more music creation
  • Synthesizing better quality sounds using artificial intelligence and the Karplus Strong algorithm
  • Adding connectivity and features to share recordings
Share this project:

Updates