Automator to turn off
Automator to play
Often times, you want to listen to music with your friends or throw a party, but the guy with the playlist is not around to click the play button. With this smart speaker you can start the music with your voice. What if the party is over? Just turn on the lights and the music will stop. The quality of a party is largely connected to the quality of music played. Many times, the person whose phone is connected to the speaker is preoccupied, resulting in terrible music being played. Furthermore, when the party has to end, the person playing music may not be around to stop it. With the smart speaker, control is given from one person to many people, allowing for better music management at any event.
What it does
The speaker is equipped with light sensors that will identify when the lights are on and then will automatically turn the speaker off. We also used shell scripts code in order to activate the serial capabilities of Arduino, and then integrated it with MAC’s Automator and dictation in order to enable voice recognition. We can play multiple songs, by saying “Play song” and turn the speaker off by saying “Turn off”. We can also say “Next” to change songs and “Cops” to force the speaker to turn off.
How we built it
- Hardware and Initial Code – Light Sensitive Speaker
In order to successfully build the Smart Speaker, we needed to find a way to send music to the speaker. We initially tried sending the music through bluetooth, but encountered multiple problems regarding the connection with a cellphone and simultaneously the speaker. Therefore, we decided to use the Adafruit MP3 shield for Arduino. This shield has an SD port and an AUX port. We transferred multiple MP3 files to the SD card and inserted it into the shield. To play multiple songs, we assigned a numerical value to each song and used Arduino’s random function to choose the song played. We connected the AUX cord from the shield to a speaker. Another advantage this hardware provided was the ability to assemble circuits simultaneously, which was integral to the functionality of our light sensor. We initially focused on the light sensitive capabilities of our speaker. We analyzed the output given by the light sensor in multiple environments in order to determine the threshold values we used for our sensor. Next, we focused on the coding aspect of our project. We downloaded the Adafruit library and then researched its many functions. When it was dark, music would begin playing through our speaker; if the lights turned on, the music would immediately stop. Next, we worked on adding voice recognition to our project.
- Adding Voice Recognition
The biggest part of our project – voice recognition – was the most difficult. As we had experience with voice recognition in MATLAB, we initially attempted integrating Arduino and MATLAB. However, MATLAB did not support the Adafruit library (which was needed to play music), so it did not work. Instead, we explored the dictation capabilities of OS and Automator applications. This required us to learn how to code in Terminal, which was challenging but quite useful. We downloaded the Arduino Serial-Utility package in order to link MAC with Arduino. For each voice command, we created a shell file in TextEdit and an Automator application to run the shell script. The shell script contained text corresponding to each voice command. Then, our applications in automator would run these shell scripts, sending the text to some location, in our case Arduino. We wrote dictation commands in OS to run the Automator application upon the recognition of each word. Finally, we wrote code in Arduino to compare the string given by voice command with our desired action. In void loop, we wrote if statements such that the Arduino would execute certain actions on the speaker (turn off, turn on, next) if the strings were the same. These steps culminated in the ability to manipulate our speaker through only speech.
Arduino Serial Package:
Challenges we ran into
One of the early stage challenges we encountered was utilizing the Adafruit shield with a circuit in a breadboard. Our first shield was flawed, which took us time to discover, so we substituted it for a working one. The second and main challenge was integrating voice commands with our speaker. We first tried with MATLAB, but it did not work for the aforementioned speaker. Ultimately we used shell scripts and Automator, with Mac dictation capabilities. This process proved quite challenging, as we had to learn how to code directly on our computer and within the scope of our operating system, something with which neither of us had any prior experience. We overcame this challenge by doing extensive internet research and conferring with multiple ESE and CIS TAs.
Accomplishments that we're proud of
We are very proud of successfully creating a project that is useful in our daily lives, despite it not being the first voice recognition product. We were always impressed by Alexa, Siri and other multiple voice recognition systems, but being able to create our own “smart” device was extremely rewarding. We are also proud of our method of voice recognition, as it required outside-the-box thinking and resourcefulness.
What we learned
Above all else, we learned that creating a fully functional product is extremely hard and time-consuming, but that marketing it is even more challenging. Our product, for instance, is not marketable because it is visually unappealing and impractical due to the problems presented by noise interference. The process itself pointed out these marketability flaws and gave us insight on what would be marketable. We also learned how to use internal coding operations on our computers, which could be quite useful in the future. For instance, we are now able to use Automator and dictation commands to perform operations on our computer. We also learned how to use Terminal, which could be important in future coding projects where we need to interface the inner workings of our computer and some third party software. We believe this final project was very rewarding, as it gave us a hands on understanding of product development and the challenges engineers face when trying to market a product in addition to vast technical knowledge.
What's next for Smart Bluetooth Speaker
The main flaw with the speaker is its inability to overcome noise interference. For the speaker to be practical and marketable, we must resolve this issue. Second, making this project wireless would be a significant improvement. We attempted this, but ran into problems and could not find a solution within the time constraints presented. Lastly, we need to clean up the look of the project, as it is presently unattractive and therefore unmarketable. The smart speaker needs to be improved and developed in order to compete in the market with Tech giants, such as Amazon, Google and Apple.
Full Report with Code
SHELL SCRIPTS CODE
This works in the following way, according to the Arduino Serial Library: it first connects with Arduino, and then send a string (-s) with the value 1 or 2. 1 will be used to turn on or change song, and 2 will be used to turn off.
./arduino-serial -b 9600 -p /dev/cu.usbmodem1431 -s 1
./arduino-serial -b 9600 -p /dev/cu.usbmodem1431 -s 2
Then we used automator's Run Shell Script Functionality to run each code. We also created dictation commands to run each Automator application.