Our idea came from a combination of simultaneously being in band at a young age, as well as the love of incredible games such as The Legend if Zelda: The Wind Waker. In The Wind Waker, Link uses a conductor's baton that creates music when he moves it in different directions. This also summons the wind gods (Cyclos and Zephos) to create extremely powerful gusts of wind. We aimed to do one of these two tasks. Considering the history of being in band (And also questioning the feasibility of controlling mother nature in 13 hours), we wanted to conduct music with this type of baton. Finally, we can be the conductors (and heroes of time) that we have always wanted to be.
What it does
Our baton, Zyphr, integrates acceleration data recorded from an 6-DOF Accelerometer/Gyroscope to calculate the user-desired BPM (beats per minute) and volume. BPM is determined by changes of direction and volume is determined by magnitude. These numbers are sent to a program that uses quasi-real time audio warping. It uses dynamic pitch and range adjustment to prevent pitch distortion. This means the BPM and volume is changed without actually impacting the pitch. All movements are recording by electronics inside of a wind waker-like baton housing. There is a button on the housing that allows for audio playing and pausing.
How we built it
For the physical device, we started by CAD-ing every electronic component we were using. Using these, we CAD-ed an electronics plate to hold a PCB and lock into a larger aesthetic housing. The electronics plate had custom holders to keep everything together. We used a NodeMCU to create an internet of things access point to communicate between the baton and the computer. The accelerometer data uses threshold values to stop slight hand movements from affected the accuracy of the data. We created a python library that utilized open source software and created a live audio mixing program. We created a system of chunking audio to make the audio adjustments more frequent.
Challenges we ran into
When designing our PCB in EagleCAD, we discovered the files we had for our NodeMCU and SparkFun device or incorrect. We spent a significant amount of time trying to fix this, before deciding to move to a direct wiring system. Everything still fits within the designed housing. We also found there a short gap in playback during audio processing. We attempted to use multithreading to pre-process the data, but it proved too complicated in the allotted time. The 3d prints for the housing also took longer than expected, making progress a little slower. This also meant we had to lose the aesthetic extending rod on the device. We also ran into a couple of slight tolerance issues that were solved with a combination of sandpaper and super glue.
Accomplishments that we're proud of
We were able to create an electronics system that turns motion into a BPM and volume level. We also found a way to take audio files and adjust the BPM while keeping pitch constant. The custom housing fit the electronics extremely well, and had good tolerances. We were able to create and almost implement everything we wanted to have even through our difficulties.
What we learned
Our team went from knowing virtually nothing in relation to audio technology, to building a quasi-real time audio stretcher that preserves pitch and audio bitrate, which turned out to be incredibly difficult but solvable. On the hardware side, we gained experience in UX design and rapid prototyping, by developing an ergonomic instrument in an extremely short amount of time. Electrically, we developed significant skills toward custom PCB creation, which only barely missed production due to time constraints.
What's next for Cyclos
We are all still very invested and interested in this project. We will continue to work on this project and polish the difficulties that we ran into. After all, would the hero of courage give up so easily?