HUDson

Renders of our tech
More renders
Our Logo
GIF
animation of our neck band
GIF
Animation of our wrist module
Image of our neckband
Image of our wriststraps
Image of our glasses
demo image
Debugging

Inspiration

While contemplating the difficulties that people with hearing deficiencies may have, we tried to relate to their experience and thought of playing video games with no sound on. We realized that often times losing a sense like hearing is detrimental to gameplay. To solve this, games implement an HUD, showing visual feedback for sounds. HUDson aims to implement this solution in the real world.

What it does

HUDson are a series of wearable tech that work in tandem to make sound more tangible for those with hearing loss. Our neckband senses audio in all directions, pinpointing both the direction and type of sound. Our AR glasses and vibrating wristband then display and give tactile feedback to our user.

How we built it

We 3d printed many of our parts, and connected each piece of hardware over wirelessly using ESP32 microcontrollers. Central processing is done on our computer and data is sent to our wristbands and an android app to display on our AR glasses.

Challenges we ran into

We had many hardware issues, from minimizing latency to connecting several devices to bricking a microphone, we had to make adjustments to our sensor input and hardware several times. We ultimately adapted our product to a satisfactory prototype. Working with audio processing in 3D space also brought many mathematical challenges that we can still improve on.

The biggest challenge of this project was the reliance we had on certain parts being available. We needed custom audio input processing, custom visual display on glasses, and novel processing that lacked existing libraries. Our microphones suddenly broke down, we lacked a USB hub for android debug data, and we struggled with wireless inconsistencies.

Accomplishments that we're proud of

We are proud that we were able to find workarounds and find the direction sound comes from. From ideation to planning to production, we were able to make a functional product with a clean aesthetic. A big struggle we've had with hardware hacks in the past was the mess that came with rapid prototyping, this time we aimed high and designed a project that would be minimalistic in design, with the goal of having no visible wires.

Learning from our past experiences, we planned for many points of failure of our project. We knew that audio interfacing with the microcontrollers were likely to fail, so we brought backup mics. We knew there was a risk using LiPo batteries with the microcontrollers so we brought a power bank.

What we learned

This hackathon taught us to be extra careful with our work, especially when using precious and limited resources. We learnt to prepare the for the worst, as having a backup plan saved us from a disfunctional prototype. Hardware-software integration is difficult, and many factors need to be checked to ensure nothing is broken. We also learnt resilience. Our original plan failed, but we quickly readjusted and returned back on track as soon as we could

What's next for HUDson

The current HUDson serves as an early MVP and proof of concept. Going forward, we envision HUDson with the following improvements: Custom PCB integration for the glasses, wristbands and microphones, eliminating the neckband, and creating a smaller form factor. With better software for tone detection, and being able to log conversations as scripts, we hope to further empower those with hearing impairments.