Inspiration
We were inspired by the need to make XR environments more accessible, especially for individuals with sensory impairments. In particular, we were motivated by vehicle collision-avoidance systems, which use proximity sensors to alert drivers of nearby obstacles. This idea of using haptic feedback to guide users' spatial awareness and prevent collisions resonated with us. We also drew inspiration from wearable haptic devices in gaming and rehabilitation, where physical feedback enhances user interaction and immersion. Our goal was to adapt this technology to create an inclusive experience in virtual environments, making VR accessible to individuals who have hearing or vision impairments.
What does it do?
Spidey Sense is a haptic headband designed to deliver spatial and directional feedback within XR environments without relying on audio or visual cues. It provides precise, real-time tactile feedback to guide users through virtual spaces. The headband features pancake vibrating motors that trigger different vibration patterns based on proximity and direction. This allows users to navigate the XR environment by "feeling" their way around, making it especially beneficial for individuals with auditory or visual impairments. The feedback is customizable, with users being able to adjust the intensity of the vibrations through a potentiometer.
How did we build it?
We built Spidey Sense by integrating a combination of hardware components, including the ESP32 microcontroller, DRV8833 motor drivers, pancake vibrating motors, and the Meta Quest 3 VR platform. The ESP32 serves as the central processor, communicating wirelessly with Unity to receive spatial data. Unity tracks the user’s position in the XR environment, calculates the relevant directional information for each motor and streams it to the ESP32. The motor drivers control the pancake motors, providing accurate and responsive vibration feedback. The system was designed to be lightweight and wearable, with all components housed in a comfortable headband. We used various tools such as Arduino IDE, Unity, Meta Haptic Studio and Solidworks to build and test the system.
The Challenges we ran into!
One of the challenges we encountered was related to the hardware complexity in controlling multiple vibrating motors. Initially, we aimed to integrate 6 pancake motors into the system, which would provide feedback in 6 distinct directions. To achieve this, we used 3 DRV8833 motor drivers, as the ESP32 microcontroller alone could not supply enough power to drive all 6 motors directly. While functional, the complexity of managing 3 motor drivers felt excessive for controlling just 6 motors, making the hardware setup unnecessarily bulky and complicated. We originally designed it to use 8 motors to represent 8 distinct directions of haptic feedback. However, this presented another significant hurdle. The ESP32 simply couldn’t handle the additional load required to control 8 motors simultaneously. Adding a second ESP32 just to manage two motors felt like an over-engineered solution, and the setup became unnecessarily complex. This would require careful synchronization between two microcontrollers, leading to additional challenges in real-time communication and control. Ultimately, we had to scale back our design to 6 motors, as controlling more motors with a single ESP32 proved to be too complex and impractical within the constraints of our project. This experience helped us understand the importance of balancing the number of motors with the processing power and communication requirements of the microcontroller. It also reinforced the idea that adding more hardware doesn’t always result in a better or more scalable solution, especially when it adds significant complexity to the system. Designing haptics for 360-degree detection was a new challenge for us. We had to fine-tune vibration intensity and patterns to ensure users could intuitively understand spatial feedback. We also considered using auxiliary haptics but chose to focus on the headband to avoid overwhelming the user and keep the system simple. This helped us refine the core experience for effective directional awareness.
Accomplishments that we're proud of!
Hardware: We are proud of the compact and clean finished product we developed. The electronic components fit neatly in the custom design “Hip Hub” and the wiring is organized, allowing a person to appreciate the intricacies behind our device.
We had little to no hiccups and interruptions during the judging and demoing session. During prototyping and development, we put a strong emphasis on building a device that is actually usable. Having a device that works reliably allowed us to demo it to everyone who visited our booth, judges, sponsors, mentors, and hackers.
Our hardware included a control knob that the user could use online to adjust the haptic intensity. Many users of our device really appreciated that customizability.
Software: The fact that it worked most of the time during the judging.
Project management: We are proud that the scope of the project is well-defined and that we were able to stay on track with the timeline. Everyone was able to effectively work on the project collaboratively. The documentation was updated as we went through the process.
User Experience: Haptic feedback feels awesome.
What we learned
Software: Test early, test often.
Hardware: We learned that the ESP32 micro controller is hardware limited to only 6 PWM channels. In our original design, we intended to include 8 vibration motors in the headband so that the vibration would span all eight cardinal directions. One consideration was to add a second ESP32 to accommodate 8 motors. However, given the complexity added did not outweigh its benefit and so we decided to use 6 motors.
We learned that regular USB cables can be used to transmit any signal. In our final design, the power and ground signals for all 6 haptic motors were transmitted to the motor drivers using 2 Micro USB cables.
Teamwork: We learned to trust each other. We started simple, took a lot of breaks during the time, and had a great time – which is the most valuable takeaway.
What's next for Spidey Sense
We plan to continue developing the application and demos. In the game where the player has to catch butterflies, we plan to enhance the game play by increasing the number of butterflies that spawn per wave. This can make the game even more interactive. The current version has some idle time in between waves.
From the hardware perspective, we plan to further reinforce the headband and look into integrating the vibration motors directly into the elastics of the quest headset. This would further simplify the user experience by eliminating the need for the user to carry additional items with them to use our device.
From the research perspective, we are planning to design a set of user studies to quantify and validate the impact of vibration on users.
Log in or sign up for Devpost to join the conversation.