Sollertia

Sollertia—Latin for skill with hand—is a mixed reality & data driven solution for stroke rehabilitation.

Comment

Inspiration

Traditional rehabilitation therapy is often outdated and very resource-intensive. A lot of it still relies on paper-and-pen exercises or basic repetitive movements, and it does not provide much real data. Clinicians usually do not get specific, calculated insight into motor performance.

After a stroke, dexterity and hand-eye coordination are commonly affected due to disrupted neural pathways. Recovery depends on engaging, repetitive practice that supports neuroplasticity, but many current tools do not offer that.

We wanted to provide a modern solution that is engaging for patients while giving clinicians real-time, measurable data during therapy.

The Data

Virtual Reality-Based Rehabilitation in Stroke Patients

Link: https://arxiv.org/abs/2405.13023

Shows that VR-based therapy improves motor recovery, engagement, and task performance compared to traditional rehab.

Point-Cloud-Based Grasping for Soft Hand Exoskeletons

Link: https://arxiv.org/abs/2504.03369

Uses fingertip force-sensing resistors (FSRs) to measure pressure, timing, and motor initiation, validating low-cost force sensors for rehabilitation metrics.

What it does

This application has three core layers working together.

1. Unity XR Experience

A Unity-based XR experience where patients going through stroke rehabilitation practice hand-eye coordination and fine motor control. Visual targets appear in the virtual environment, and patients interact with them using controlled finger presses, allowing repetitive but engaging motor practice.

2. Clinical Dashboard

A clinical dashboard that captures session data from the XR experience. This includes metrics such as:

  • Reaction time
  • Press frequency
  • Force magnitude
  • Task completion trends

Clinicians can review post-session performance and track progress over time instead of relying only on observation.

3. Wearable Hardware

This glove uses force-sensing resistors (FSRs) to capture finger pressure, with a proof-of-concept GSR layer designed as a virtual circuit to explore physiological signals related to engagement and effort

Together, this system supports both patients and clinicians by turning therapy sessions into measurable, data-driven experiences.

How we built it

To build this, we created a custom hardware and software setup.

We used an ELEGOO UNO R3 with force-sensing resistors (FSRs) placed on the fingers to capture pressure input, specifically the magnitude of applied force during interaction. This allows us to measure dexterity and control in a more quantitative way.

We also designed a proof-of-concept GSR integration using a virtual CJMCU circuit to explore how physiological data could be interpreted alongside motor performance.

Sensor data is sent using websockets for real-time communication into the application. On the software side, the incoming data is mapped into an interactive environment where the patient’s physical input directly affects the experience.

A dashboard was made in rust and included for clinician view, allowing them to visualize and log data in real time to influence patient outcomes. The index and middle fingers were chosen because they are both functionally important for dexterity and effective points for collecting reliable GSR signals.

Challenges we ran into

Unity XR (mixed reality --> headset) was a challenge.

Accomplishments that we're proud of

We are very proud of creating a project that will help tons of people who suffer with recovery from neurological damage. The feeling of knowing we created something that can make a huge difference within the community is a feeling like no other.

What we learned

We learned that recovery from neurological damage is not just a few months of exercises. It is often a long, painful, and frustrating process. When progress does not visibly match effort, it can be extremely discouraging.

That is why we chose to gamify rehabilitation. Engagement, confidence, and feedback matter just as much as repetition. We enjoyed building something that will support patients emotionally as well as physically during recovery.

Teamwork really does make the dream work.

What's next for Sollertia

  • Embedded system optimization for a more compact, wearable design
  • Integration of additional sensors such as PPG for heart rate and fatigue analysis
  • More advanced data analysis to identify long-term recovery patterns and personalize therapy plans
  • Expansion to all fingers

Built With

Share this project:

Updates