Orca

• 

  Architecture

• 

  Settings

• 

  Main menu

• 

  Agents page.

Inspiration

I have spent a long time building projects for people with disabilities. Somewhere along the way I stopped thinking about the technology and started thinking about the people using it. Tasks the rest of us never notice can take real effort, patience and courage. And much of their recovery happens far from any clinic, at home, with almost nobody watching.

That gap stayed with me. A person does their exercises alone, and a clinician gets almost nothing to go on. A wearable can count repetitions, but one noisy sensor cannot tell a weak hand from a resting one. Worse, an alert that turns out to be wrong makes people stop trusting the system, and the people who need help the most are the ones who can least afford to be misled.

So I wanted to build something that treats them with care. It should make rehab feel like a game instead of a chore, it should show its reasoning, and it should say "I am not sure" when it is not. That is Orca.

What it does

Orca turns FreeWili OG devices into a two-handed rehab game station. Two OGs are worn as controllers (tilt sensors), a third is the screen and button pad, and a laptop runs the games. Four Fetch.ai agents handle what the sensors report:

• Filter drops readings that cannot be real (glitches, no face in view) and compares the two hands, and the face against the person's own calm baseline.
• Check fact-checks a possible imbalance four ways before calling it real: enough data, persists across windows, unusual for this person, and other signals (face, pain) agree.
• Decide turns the verdict into a difficulty recommendation with a stated rule. A half-checked finding can never make the game harder, and a verified concern eases it.
• Act saves the round and the clean readings (local queue, then Supabase) and states the recommendation.

Six games sit on top of this, each built around a different movement: Beat Flick, Rhythm Flick, Steady Hand, Dial, Brick Break and Colour Reflex. Every agent step is visible on the OG screen, with a page for each agent, and a gateway agent answers questions about the chain in ASI:One chat.

How I built it

• FreeWili OG firmware in C (two RP2040s) as a USB display and button pad. A Python host runs the games, reads the two hand OGs, and mirrors the game to the screen OG.
• Four Fetch.ai uAgents, one per stage, each on its own port with its own Agentverse mailbox and chat-protocol profile. The stages hand work to each other directly on the laptop.
• Supabase for rounds and readings (insert-only key).
• A simulated patient (seven sessions over two weeks) that plays through the real agents, so the verdicts shown are produced by the pipeline, not typed in.

Challenges I ran into

• No IMUs. Motion tracking normally uses an IMU, and I did not have one. The FreeWili OG has a 3-axis accelerometer, which gives tilt but no heading or rotation speed. I added a BMM350 3-axis magnetometer on the OG's I2C header through an Orca module and built a motion estimate from the two: tilt from gravity, heading from the magnetic field, a quick calibration from a few slow turns that removes the magnetometer's large fixed offset, and a check that tells turning the hand apart from sliding it sideways. This is not a full IMU. It gives no position, has no gyroscope, and metal or magnets nearby throw off the heading.
• Linking three devices. The two hand OGs and the screen OG have to act as one system. I built a link between all three, coordinated through the laptop over USB: each OG gets a role (left hand, right hand, screen) set by pressing a button on its own screen and saved, and the screen OG can show pictures and take buttons while the hand OGs stream tilt. It is a hub-and-spoke link through the laptop, not a wireless mesh.
• Telling a weak hand from a resting hand. An idle hand once looked like a severe imbalance, until I required both hands to move enough before comparing them.
• Four agents, four Agentverse profiles. Getting a four-agent chain to run as separate Agentverse agents on Windows, each with its own port and mailbox.
• Keeping the agents honest. They are only allowed to make the game safer.
• Doing it alone. Firmware, hardware, games, agents and the database, all in one weekend, with no one to split the work with.

Accomplishments I am proud of

• Motion tracking without an IMU. An accelerometer and a magnetometer together give the games tilt and heading, tell a turn from a slide, and let my tester tell apart held still, spun flat, tilted, shaken and a magnet nearby. That is what the games and the agents need to read a wrist or forearm.
• A chain whose every step can be read on a device screen. Each of the four agents has its own page on the OG.
• A rule that a half-checked finding never makes the game harder.
• A patient story the agents handle correctly. Noted, watch, concerning and recovered, each ending in the right verdict.
• Three OGs working as one system, with roles set from their own screens.

What I learned

• A verdict on its own is not worth much. Showing the checks behind it is what lets someone trust it, or rightly not trust it, and "not enough data" turned out to be one of the most useful answers the system gives.
• An idle hand looks a lot like a weak hand until you make sure both hands are actually moving. Most of my false alarms came from the data, not the logic.
• You can get surprisingly far without the "right" hardware if you are honest about what your sensors can and cannot see.
• Keeping the agents limited made them better. They can only ease the game or leave it alone, and that one rule made them simpler to build and easier to trust.

What's next

Validate the thresholds with clinicians (they are placeholders today), test with real patients under proper oversight, and add more games. Try the straps on other limbs: today the OGs are worn on the hand or forearm (the games are written for wrist and forearm movement), and legs or ankles would need new games and testing. Add real IMUs for full orientation tracking.

Honest limits

Research prototype. Thresholds are placeholders and not clinically validated yet.

Built With

• agentverse
• asi:one
• c
• fetch.ai-uagents
• freewili-og
• python
• rp2040
• supabase
• tkinter