Praxis

Praxis is a QNX-powered rehabilitation tool that turns a simple tracing exercise into measurable, AI-powered insights about a patient's accuracy, stability, and movement quality, session over session.

Comment

Inspiration

Stroke, Parkinson's disease, and multiple sclerosis (MS) can all impair fine motor control, making everyday tasks much more difficult. Patients are often prescribed tracing exercises to rebuild or maintain these skills. Still, between clinic visits, there is rarely a quantitative way to measure whether their performance is improving, staying the same, or declining. Therapists rely on observation, while patients may struggle to stay motivated because they cannot clearly see their own progress. That problem became the inspiration behind Praxis. Our team consists of two biomedical engineering students and two software engineering/computer science students, and we wanted to build something at the intersection of healthcare, embedded systems, and AI. We kept coming back to tracing exercises because they are simple, inexpensive, and already familiar in rehabilitation. But a completed drawing only tells half the story. It shows what the participant drew, but not how they moved while drawing it. Two traces can look nearly identical even though one was completed with smooth, controlled movement and the other involved tremor, hesitation, or repeated corrections. We wanted to capture both sides of that story by measuring both the final tracing result and the movement that produced it.

What it does

Praxis is a QNX-powered rehabilitation assessment platform that helps therapists objectively measure upper-limb motor performance during standardized tracing exercises. Participants complete a tracing task using a pen-like device while the system records both the completed tracing and the movement used to produce it. By combining camera and motion sensor data, Praxis measures four key aspects of performance:

  • Mean Deviation: How closely the user's trace follows the reference path.
  • Completion Time: How long it takes to complete the tracing task.
  • Task Completion: The percentage of the tracing task successfully completed.
  • Hand Tremor (RMS Angular Velocity): The amount of involuntary high-frequency movement during the task.

Rather than evaluating a participant against a general population, Praxis compares each session with that participant's own previous attempts, allowing therapists to objectively monitor changes in performance over time. After each session, results are automatically saved and displayed through an interactive dashboard, where therapists can review historical trends, compare sessions, and track progress. To make the results easier to interpret, Praxis also generates a concise summary that explains how the participant's performance has changed between sessions.

How we built it

Praxis was built on QNX 8.0 using a Raspberry Pi 5, a Raspberry Pi Camera Module 3, and an MPU6050 inertial measurement unit (IMU). To create a portable tracing device, we attached the IMU to a Raspberry Pi that was securely mounted alongside a pen so the entire assembly moved naturally with the participant's hand during tracing. To ensure consistent image capture, we mounted the Raspberry Pi Camera Module 3 above the tracing surface in a rigid overhead box, providing a repeatable top-down view of every completed tracing task.

During each session, the IMU continuously records motion data. Once tracing is complete, the camera captures a high-resolution image of the finished trace. Our image-processing pipeline analyzes the captured tracing by comparing it with the reference template to calculate mean deviation, task completion, and completion time. At the same time, the IMU data is processed to compute hand tremor using the RMS angular velocity. All of this measurement work runs on-device on the QNX real-time OS, so the core assessment keeps working even without a network connection.

Directly on the QNX device, we also run a small quantized language model, Qwen2.5-0.5B via llama.cpp, that generates a plain-language explanation of the finished session locally. Its output is schema-constrained and every number is validated against the deterministic result, with an automatic fallback to a template on any failure or timeout, so the AI can add insight but can never alter a measurement.

The completed session is transmitted through a REST API to a FastAPI backend, where the incoming data is validated before the measurements, trace geometry, captured-frame quality information, and metadata are securely stored in MongoDB Atlas as a longitudinal participant record. A React web dashboard retrieves this data to visualize participant progress and compare performance across repeated sessions.

To turn those repeated sessions into clinical insight, Praxis uses FreeSOLO, a language model we fine-tuned ourselves: starting from Qwen3.5-4B, we ran supervised fine-tuning (SFT) followed by GRPO reinforcement learning on a corpus of deterministically generated rehabilitation session comparisons, then deployed the resulting model on Modal behind an OpenAI-compatible endpoint. The backend calls it through a strict adapter that validates every output before storage. FreeSOLO receives verified measurements and historical comparisons and explains whether a participant's performance has improved, remained stable, declined, or shown mixed changes.

Finally, to improve accessibility, we integrated a Gemini-powered assistant that helps users, particularly older adults and less technical users, navigate the dashboard, understand its features, and interpret what is currently on screen, always deferring clinical judgment to the therapist.

Challenges we ran into

One of our biggest challenges was designing a hardware setup that could reliably capture both the completed tracing and the participant's movement. We also had to work within the limited materials available at the hackathon and pivot our original idea from sleep tracking to upper-limb rehabilitation. In hindsight, that change turned out to be a blessing in disguise, leading us to a project that better combined our interests in healthcare, embedded systems, and AI. Our initial approach was to continuously track the pen with the camera, but we quickly ran into issues with camera positioning, hand occlusion, and maintaining a consistent field of view. We redesigned the system so the IMU continuously measured movement throughout the task while an overhead camera captured a single high-resolution image after tracing was complete. This simplified the computer vision pipeline, improved reliability, and produced more consistent measurements despite our limited hardware and time constraints.

What we learned

This project taught us that building a healthcare application requires much more than developing an accurate AI model. Reliable hardware design, sensor synchronization, calibration, data quality, and system architecture all play equally important roles. We also learned how valuable rapid prototyping and iterative hardware design can be. Several of our best design decisions came from adapting our approach after testing early prototypes.

What's next for Praxis

Our next goal is to expand Praxis beyond a single tracing exercise by introducing multiple tracing patterns and difficulty levels that target different aspects of fine motor control. Ultimately, we envision Praxis becoming a scalable rehabilitation platform that enables data-driven monitoring in hospitals, clinics, rehabilitation centers, and home-based therapy.

Share this project:

Updates