Air mouse

Air mouse prototype

Inspiration

The inspiration for the Air Mouse stemmed from the limitations of traditional human-computer interaction. Standard mice are tethered to flat surfaces and physical space, creating barriers for dynamic presenters and users with accessibility needs who may find desktop-bound peripherals restrictive. We wanted to create a solution that prioritizes convenience, allowing for precise navigation through natural, hand-held gestures

What it does

The Air Mouse is a wireless peripheral that maps 3D hand movements to 2D screen coordinates with high precision and zero drift. Intuitive Control: Users "Nod" for vertical movement and "Twist" (Yaw) for horizontal movement.

Physical Feedback: Integrated tactile buttons on the device allow for standard Left and Right clicking.

Stability: The device remains rock-solid at rest, ensuring a plug-and-play experience without the "phantom movement" common in basic inertial sensors.

How we built it

The system architecture bridges hardware sensing with software-level OS control.Hardware Stack: Built on the ESP32-S3 microcontroller and the LSM6DSOX 6-axis inertial sensor.Communication: Used the NimBLE Bluetooth Stack for low-latency, 100Hz wireless data transmission to a host PC.Signal Pipeline: The Python backend processes raw data through a Deep Calibration Protocol and a Kalman Filter to ensure smoothness.

Challenges we ran into

Developing a high-precision inertial mouse presented several engineering hurdles:

Sensor Noise & Bias: Manufacturing imperfections in the gyroscope led to constant cursor "drift". We solved this by implementing a 300-sample calibration window at boot to calculate the static bias level.

Axis Crosstalk: Mechanical force from clicking a physical button caused the hand to shift, moving the cursor off-target.

Stabilization Window: We developed a logic gate that "locks" the cursor position for 150ms during a click event, ensuring pinpoint accuracy during mechanical interaction

Accomplishments that we're proud of

High-Fidelity Control: Achieving 100Hz high-speed polling, which minimizes Riemann-sum error and results in a near-lagless user experience.

Robust Signal Processing: Successfully implementing a Kalman filter to manage stochastic noise, turning a noisy raw signal into a smooth, usable cursor.

Team Collaboration: Coordinating a multi-disciplinary stack involving C++ for the ESP32, Python for the backend, and the mathematical implementation of sensor fusion

What we learned

Embedded BLE: Managing the NimBLE stack on the ESP32-S3 for stable data throughput.

Inertial Math: Deepening our understanding of 3D coordinate frames and the numerical integration required to transform rad/s into screen pixels.

User Interface Design: Understanding that "convenience" in hardware requires accounting for human factors, such as mechanical jitter during button presses