Compact Action Piano Visualizer Piano Layout

Pillar:

Best Technical

Inspiration

Piano lovers that have no access to a physical one or are not affordable for one is the reason why we're building this project, especially for piano major students who have less access to lab compared to other science or engineering major. We're making a portable piano that can fit in a backpack, connecting to digital devices via bluetooth for a visualizer and speaker.

What it does

• Connect digital devices via Bluetooth by scanning device's name or server • Running the web server while blocking anything might interrupt to listen to the piano notes • Displays MIDI pitch into natural piano sounds • Visualize a piano keyboard (7 lower keys and 5 upper keys) • Displays the same keys and animations as the connected physical piano

How we built it

• Python Backend: Uses Flask and Bleak for background BLE scanning and connection handling, Flask-SocketIO keep track of the key's state • HTML Frontend: Makes a light-purple themed visualizer built on HTML5 Canvas and CSS Flexbox. Use requestAnimationFrame in rendering loop to make the note pressing animation. Programs the piano pitch using Web Audio API matching standard calculations. • Model and design a custom piano layout that is compact. • 3D printed the model and solder wire connections. • Use nickel plate as customer switch and using compile mechanism for the key spring • Use ESP32 to handle Bluetooth connection as well as the logic of the keyboard

Challenges we ran into

• In bleak, client.start_notify() expects a callback function with callback(sender, data), however, socket.emit() is synchronous when called from standard Python threads. We were facing difficulty while figuring out an approach to call async code inside. • We handled events across threads incorrectly • We had problem running the connection retry continuously without blocking the running one • Figure out how to detect sliding fingers across keys • Physical item 3D printing takes time, the initial circuit was wrong due to mistakes in electrical diagram • Complicated wire connection

Accomplishments that we're proud of

• Successfully built a piano visualizer website with Bluetooth connection to digital devices • Successfully made a 3D physical product, working wirelessly with a device • Create our own customer tactile switch

What we learned

• Use a sleep interval in a while True loop ensures backend is resilient • Advanced asynchronous Python uses asyncio and Bleak with WebSockets • 2D HTML5 Canvas rendering techniques for real-time interaction and animation display • Compliant mechanism to create the spring for the keys • Learned how make the eps32 communicate with a device via Bluetooth

What's next for Compact Action Piano

We're running the project by running multiple files at the same time, to make sure the web server is easy to use for the users, we'll figure out how to make the device run the files in the background and the only thing displaying is the visualizer