Con Anima

Inspiration: Music is all around us. From concerts of our favourite artists, to small performances at a schools art festival. But we often don't get the opportunity, or have the foresight to learn how to get involved in music early. We felt that people often disengaged from music because they felt they had no way to truly express themselves with music.

What it does: Con Anima generates music using a suite of bio-sensors and environment vision processing. You choose the way the music flows from within you.

How we built it: Con Anima is composed of a mobile/web application an Express + Socket.io back-end, and a Raspberry Pi + ESP 32 powered Bio Suit that uses Gemini Processing on image data and a Heart Rate + Galvanic Skin Response + Body Temperature Fusion Sensor.

Challenges we ran into: On the first day, we planned on using ECG nodes to be able to accurately measure the heart rate. Unfortunately, we were provided the wrong driver board, and spent around an hour trying to get it to work. We also ran into issues regarding ESP32 analog input, spending a large deal of time getting a usable signal before stumbling across Espressif documentation that helped us greatly. The Raspberry Pi Zero 2 W isn't powerful enough to continuously run image processing, thus multiple prompting systems were used to decide when image processing would be run to optimize resources. 3-D print was supposed to be printed by the night of the 14th, but the print took unexpectedly long so the hardware assembly had to be done in the morning of the 15th

Accomplishments that we're proud of: Usage of experimental Google Lyria Real-time due to a lack of documentation and online resources Integration between all our components and systems Denoise-ifying sensors to a very accurate level A successful demo

What's next for Con Anima: Higher quality sensors to increase reliability (heart rate --> smart watch, camera --> Meta Ray-Bans) Bluetooth instead of reliance on websockets to transmit data to reduce complexity Using CV instead of LLM to process visual data to reduce costs and increase efficiency

What we learned: We learned how to use web-sockets, real-time data processing, biometric sensors, implementation of AI, and mostly importantly the true meaning of love (Thank you Andy).