Inspiration:
I've been working on myoelectric prosthetics through my school's research program for about a year now. The thing that kept bothering me was that these AI models are completely opaque — they take in muscle signals and spit out hand movements, but nobody can tell you why. When the hand moves wrong, the user is just stuck. No explanation, no adjustment, nothing. They recalibrate and hope for the best. I kept thinking: if I'm controlling a robot with my body, shouldn't I be able to understand what it's doing and do something about it in real time? What it does
Origami xAI is an explainable AI system for prosthetic hand control. It sits on top of a Recursive Multimodal Sparse Transformer that reads 14 forearm muscle signals (sEMG) and predicts 35 joint angles of a robotic hand.
The three main features:
A sensitivity dial (λ) — basically a volume knob for the AI. When your muscles get tired, you drag it left and the AI uses more of its processing power to compensate. For simple movements, drag right and it saves compute. No recalibration, no taking the device off.
Channel attribution — after each prediction, a bar chart shows which muscles the AI actually listened to. Turn on the fatigue simulation and you can watch those bars shift in real time as the AI starts relying on different muscle groups. The system fails loudly instead of silently.
Dynamic model unfolding — the model has 4 processing blocks but doesn't always use all of them. Simple movements use 1 block, complex ones use all 4. Same model, different folds. Like origami. This is inference-time adaptation only, no weights change, so it actually saves battery on a real device.
How we built it:
The model is a Recursive Multimodal Sparse Transformer (RMST) with 4 RMCABlock layers, gated recycling connections, TDC convolution, and axial attention. The input is sEMG [B, N, 25, 14] plus a kinematic seed [B, 1, 25, 35], and it outputs a continuous joint trajectory [B, N×25, 35].
The origami modification just slices the block stack in the forward pass: self.rmca_stack[:recursion_depth]. Tier 1 uses 1 block, Tier 2 uses 2, Tier 3 uses all 4. No retraining needed.
For explainability, I used gradient-based channel attribution — set requires_grad on the EMG input, run forward, backprop, then normalize the absolute gradients to 100%. Same principle as Integrated Gradients.
Signal complexity is measured as H_data = std(log(var(EMG, dim=time) + ε), dim=channels), which is a validated muscle fatigue indicator from Phinyomark et al. (2013).
The fatigue simulation uses a 4th-order Butterworth low-pass filter at 0.4 Hz to model spectral compression from prolonged muscle use (De Luca, 1979).
For validation I used MuJoCo physics simulation with the Shadow Hand model (20 DOF), checking joint limits with 5° tolerance and running dynamic self-collision detection. I also applied 2:4 structured sparsity from NVIDIA's research for a 19.35% model size reduction.
The dashboard is built in Streamlit with Plotly charts.
Challenges we ran into:
The biggest one was something I completely did not expect. When I first looked at the joint angle comparison graphs, the predictions looked terrible. I was genuinely panicked. But then I fed those same predictions into MuJoCo and the 3D hand movement was actually smooth and physically realistic. The individual joint numbers look noisy, but the overall hand posture cancels out those small errors. That was a huge learning moment about the difference between per-joint accuracy and actual physical feasibility.
The other frustrating one was a calibration issue where the H_data values from my test data were almost constant across all trials. That made the lambda slider look completely broken — it didn't matter where you dragged it, the tier never changed. I had to rethink how the threshold scaling worked to make the tier selection actually respond to the slider.
Accomplishments that we're proud of:
Getting 79.4% physical feasibility in MuJoCo validation. That means 79.4% of our predicted joint angles are physically possible on a real Shadow Hand robot.
The channel attribution actually working in real time — you can toggle fatigue on and watch the muscle importance bars visibly shift. That was the "it actually works" moment.
The fact that the whole system runs in a Streamlit dashboard that anyone can interact with. The lambda slider, the tier changes, the attribution, the fatigue toggle — it's all live and responsive.
What we learned:
That joint angle accuracy and physical feasibility are very different things. A model can look bad on paper but produce physically realistic movements, and a model can look good on paper by just predicting the same safe pose every time (which is what the LSTM and Ridge baselines actually do).
That explainability in prosthetics isn't just nice to have — it's a practical tool. When you can see which muscles the AI is relying on and adjust the processing depth, you have actual agency over the device.
That inference-time compute adaptation (the origami unfolding) can work without any retraining or fine-tuning. The same weights, just different amounts of processing.
What's next for MyoOrigami xAI:
Real-time EMG hardware integration, right now this runs on pre-recorded data. The next step is connecting it to actual EMG sensors and getting the inference latency down for real-time control.
User studies to see if the lambda dial and attribution display actually help prosthetic users in practice, not just in theory.
Expanding the validation across more trials and movement types, and exploring whether the origami concept generalizes to other adaptive neural network architectures.
Log in or sign up for Devpost to join the conversation.