Haptic Gaming Mouse

Setup for Functioning Haptic Gaming Mouse
Custom Designed Stepper Motor Casing

Inspiration

Accessibility and immersivity in the PC gaming industry has always fallen below industry standards as gaming consoles and handheld devices are becoming more ergonomic and haptic-based. Consoles, such as the PS5, have implemented haptics in controllers to ensure that users can be more immersed in the games they play, delivering tactical responses from user gameplay. Gaming mice have little to no feedback, only providing a solid click when a user presses either button. In addition to this, PC users experience Carpal Tunnel Syndrome, as well as other joint/nerve issues at a disproportionate rate due to poor ergonomics and prolonged gameplay. With low immersivity, outdated standards, and potential health hazards; the standard PC gaming mouse needs a fresh redesign.

What it does

Our haptic gaming mouse add-on uses a force sensor to receive and analyze clicking force from a user. This allows the mouse to provide haptic-based feedback to the player’s wrist proportional to the intensity of the user’s interaction. From FPS-shooting to Minecraft-digging, the possibilities are endless!

The addon attaches via an adhesive on the bottom of the custom designed, 3D printed casing and features a compact design that will easily allow any gamer to switch the add-on between mice. It also features an energy-efficient automatic power-off feature that turns off the haptic system after a period of inactivity.

The add-on also implements a smart SMS function through Twilio to ensure the player is minimizing health risks while gaming. The features implemented through Twilio notify a player when their gaming session exceeds a healthy time limit, as well as when the player is clicking with excessive force; which is also visually displayed through an LED indicator. To help minimize energy consumption, it can also notify a user when the device automatically powers off due to inactivity.

How we built it

For the hardware, we utilized an ESP32 microcontroller, which allowed for efficient energy consumption and internet capabilities. A force sensitive resistor, functionally a force sensor, was implemented on the left-click area of the mouse. The values from the sensor (which are collected on a scale from 0-1000) are mapped to motor rotation speeds (on a scale from 300-1000). The motor implemented for this project was a stepper motor that provided haptics to the user when in motion (a lower motor speed corresponds to a deeper vibration). The step motor is attached to the back of the mouse using a custom designed, 3D printed mount and the vibrations from the motor provide haptics to the user’s wrist. An LED was also implemented in the circuitry, and indicates the user when motor speeds go below 400, which is indicative that the user is pressing too hard.

For the SMS service, we implemented a custom Twilio ESP32 library to interact with the ESP32 microcontroller. This uses the microcontroller’s WIFI connection to allow users to interact with Twilio over a network SSID. The goal of the SMS implementation is to send text messages to the user’s device while using a Twilio-generated number. The program begins by connecting the device to WIFI and creating a Twilio object using the Twilio API with an account SID and Authentication Token. Twilio waits for a condition (gaming for too long, pressing too hard, inactivity) before sending an SMS notification to the user’s phone number.

Challenges we ran into

One of our biggest challenges was powering our device. Since we did not have access to batteries that could provide enough voltage and current to drive our stepper motor, we had to learn how to safely use a programmable DC power supply (making sure both the power supply and Arduino share the same ground). Additionally, we originally designed our haptic gaming mouse add-on using an Arduino Uno board, and later decided to incorporate Wifi functionality, which would allow for the device to utilize IOT capabilities. This feature implementation made us rewire our device to an ESP32 board. The new layout was much less intuitive compared to the Uno, and caused issues within our pre-existing hardware and software.

Accomplishments that we're proud of

For this project, we were proud to develop a functioning hardware project in a short time frame. Within a new setting, all team members were able to gather a variety of tools, materials, and resources efficiently in order to develop a fully-fledged product. In addition to this, the team was also incredibly adaptive to new changes/implementations. We decided to incorporate IoT functionality much later in the design process, however, all team members were able to integrate the new mechanics and software incredibly effectively. Finally, being able to incorporate hardware, software, and IoT programming within this hackathon was a great accomplishment.

What's next for Haptic Smart Mouse

For the future of the Haptic Gaming Mouse, we plan to also take inputs from the game that the user is playing so that users can also feel if something affects their players. We would implement this by using the game’s API or training machine learning models on sounds from the game so users can feel haptic feedback if they were to be affected in game (ex. Getting attacked in-game). We would also implement a custom designed PCB (printed circuit board) and a more standardized/accessible power source in the future in order to have a smaller form factor for the mouse, increasing portability.