Dextera Glove
Landing Page
Recovery Progress Analytics
Carrom Game
Piano Game
Ball Pickup Game
Bubble Pop Game
Logo

Dextera Project Story

Inspiration

Hand rehabilitation is repetitive by design, but that repetition can be difficult to maintain outside the clinic. People recovering from stroke, neurological injury, or hand and wrist trauma often need hundreds of careful movements to rebuild mobility, grip control, finger isolation, and confidence. Clinicians, meanwhile, usually have limited visibility into what happens between appointments: whether patients practiced, how well they moved, which fingers struggled, and whether pain or fatigue increased.

Dextera was inspired by that gap. We wanted to build a system that makes hand therapy feel more engaging for patients while giving clinicians objective, real-time recovery data. Instead of asking patients to repeat exercises in isolation, Dextera turns therapy into guided games powered by a smart glove.

What It Does

Dextera is a smart-glove rehabilitation platform for hand therapy. A glove with five flex sensors measures finger movement and streams data into a web platform. Patients use the glove to play rehab games, while clinicians monitor progress through a dashboard.

The platform includes:

A five-sensor ESP32 glove that captures thumb, index, middle, ring, and pinky bend data.
A serial bridge that forwards glove frames to the backend in real time.
A clinician dashboard for patients, assignments, alerts, sessions, analytics, and difficulty recommendations.
A patient portal with assigned exercises, calibration, tutorials, pain/fatigue check-ins, results, and progress charts.
Four rehab games: Ball Pickup, Finger Tap Piano, Bubble Pop, and Carrom.
Calibration flows that adapt the glove to each patient’s open hand, fist, taps, point, pinch, and flick gestures.

Dextera tracks more than just whether a game was completed. It measures reps, accuracy, failed attempts, weakest finger, adherence, pain, fatigue, and improvement over time.

How We Built It

Dextera is built as a full hardware-to-software pipeline:

Flex sensors on glove
        |
ESP32 reads raw ADC values
        | USB serial
Node serial bridge normalizes and forwards frames
        | HTTP POST
Backend API stores events and broadcasts updates
        | WebSocket + REST
Doctor dashboard + patient rehab games

The glove uses five flex sensors wired into ESP32 analog pins. Each sensor forms a voltage divider with a resistor, producing raw ADC readings from 0 to 4095. The ESP32 prints one JSON object per frame over serial.

The bridge reads those serial frames, fetches calibration data from the backend when available, converts raw values into 0-100 bend percentages, and posts events to the API. The backend stores events, sessions, assignments, alerts, calibration data, and patient records, then broadcasts live updates over WebSockets.

The frontend is a React and TypeScript app with two main experiences:

The doctor workspace for monitoring patients and assigning rehab plans.
The patient workspace for playing games and tracking recovery.

For game interaction, we combined glove gestures with camera or pointer-based movement depending on the game. Ball Pickup uses open/fist grip control, Finger Tap Piano uses per-finger tap detection, Bubble Pop uses point/pinch targeting, and Carrom uses calibrated finger flick speed for shot power.

Challenges We Faced

The biggest challenge was making raw glove data feel reliable enough for gameplay. Flex sensor readings vary by person, hand size, glove placement, and sensor bend range, so hard-coded thresholds were not enough. We built calibration flows for open/fist, finger taps, point/pinch, and Carrom flicks so the same glove could adapt to different patients.

Another challenge was preventing accidental actions during games. For example, Carrom needed a precise flow so a patient could place the striker, lock placement, aim, lock direction, pause, and then flick without the previous hand posture accidentally triggering the next step. We solved that with an explicit state machine and timed pauses between actions.

Performance was also important. Bubble Pop initially became laggy because the game was updating too much React state while camera tracking was active. We optimized the game loop, smoothed the camera cursor, and kept fast visual updates separate from heavier scoring state.

Finally, we had to make the app useful with and without live hardware connected. That meant building realistic fallback data, local result persistence, and clear calibration/test flows while still preserving the real glove pipeline.

What We Learned

We learned that rehab input design is different from normal game input design. A gesture is not just a button press; it is a physical movement with fatigue, stiffness, delay, and noise. Good rehab software needs to respect that. It should guide the patient, confirm intent, avoid accidental triggers, and make progress visible without pressuring the patient to move unsafely.

We also learned how important calibration is. The same raw sensor value can mean different things for different hands, so the system needs patient-specific baselines. Calibration turned the glove from a raw sensor device into a clinically meaningful input tool.

On the software side, we learned how to connect hardware, real-time APIs, game mechanics, and healthcare-style analytics into one cohesive flow.

What Makes Dextera Different

Dextera is not just a glove demo and not just a rehab game. It connects the full loop:

The patient practices through interactive therapy games.
The glove captures real movement data.
The backend stores and streams that data.
The clinician sees progress, alerts, adherence, and recommendations.
The patient gets feedback and a reason to keep practicing.

That closed loop is what makes the platform powerful. It turns home rehab from a black box into something measurable, motivating, and easier to support.

What’s Next

The next steps for Dextera are:

Improve the physical glove enclosure and sensor mounting for durability.
Add more clinician controls for custom rehab plans.
Expand analytics around range of motion, smoothness, fatigue, and long-term trends.
Add more games for different therapy goals and ability levels.
Improve remote deployment so patients can use the glove from home while clinicians monitor progress from anywhere.

Dextera shows how low-cost sensors, real-time software, and game-based design can make rehabilitation more engaging for patients and more actionable for clinicians.