HandyCane

Inspiration

Our design brief centers on Alex Chen, described as a biology teacher with ALS. The brief describes three details that stuck with us:

He already uses a cane, one assistive device that hasn't ended up in a drawer.
He has fallen twice, both after 6 pm, on stairs, and he's home alone until 4:30 on many days.
Between clinic visits, his progress is tracked by a questionnaire every three months. In the gaps, nobody sees how his movement is changing.

In the brief, Alex says he wants to stay "reachable and useful" and to get to his daughter's graduation without a wheelchair. We didn't want to give him another gadget to wear. We wanted to make the thing he already carries do more.

What it does

HandyCane is a smart-cane prototype with two jobs.

1. Local safety without an internet connection. Software on a Raspberry Pi reads a cane-mounted accelerometer to detect when the cane goes down, with or without an impact. It can drive a local buzzer, play spoken prompts and start an escalation countdown. The spoken prompts are voiced with ElevenLabs and pre-generated, so they are stored on the Pi and played locally with no live API call. Pressing the button cancels the alert flow. Holding the button for two seconds triggers an SOS. Detection, local logging and local audio do not depend on a phone, Wi-Fi, cloud service or AI. Remote caregiver alerts are separate: on the cane and from the backend, which re-sends the notification while an alert stays open. This requires internet access and configuration. SMS and push notifications are not implemented.

The system detects potential falls or cane drops, not confirmed human falls. It cannot tell from cane acceleration alone whether a person fell. Likewise, picking up the cane is not proof that the person has recovered.

2. Longitudinal cane-use monitoring. The cane software stores timestamped records locally. The companion app can pull those records over local Wi-Fi and upload them to the backend when online. The backend calculates cane-active minutes, cane strikes, bout duration, cane-rhythm variability and event counts. It also summarizes time-of-day activity, and reports include the person's most active hours and any shift in peak activity time versus their reference period.

The backend can generate an on-demand report covering a 14-day period, with comparisons to a previous period and a patient-specific reference period. Reports can be viewed through patient and authorized care-team workflows. An optional scheduler job can also generate each 14-day report automatically when enabled in deployment; automatic delivery to a family or clinic is not implemented.

What it is not: a diagnostic tool, a validated measure of ALS progression or a replacement for an emergency system. The sensor measures how the cane moves, not gait quality, balance or ALSFRS-R. Every generated report includes: "This report describes cane-use patterns and is not a medical diagnosis."

How we built it

Cane runtime: Python software for Raspberry Pi, with support for accelerometers and a push button. The code also supports a buzzer and local audio output.
Voice prompts: The cane's spoken prompts are voiced with ElevenLabs. We generate the audio ahead of time and store it on the Pi, so playback needs no internet connection or API call at runtime. ElevenLabs' role. ElevenLabs provides the voice for the cane's spoken prompts. The audio is generated ahead of time, so it affects only how the prompts sound. Detection and escalation logic run in the Pi software and do not depend on it.
Detection: a configurable threshold-based state machine using acceleration magnitude, tilt and stillness. In the current Pi runtime, an impact is above 2.5 g. A down event requires tilt above 60 degrees and smoothed dynamic acceleration below 0.05 g for more than 1.5 seconds. Pickup requires tilt below 45 degrees for more than one second. These are software starting points, not validated clinical thresholds.
Phone sync: HTTP over local Wi-Fi, using pull/ack and persistent per-device sequence numbers to prevent duplicate records. BLE is a future transport, not a working feature.
App: React Native, Expo and TypeScript, with patient, caregiver and clinician workflows, simulated demo mode, local storage and an upload queue.
Backend: FastAPI and SQLAlchemy, using SQLite by default. PostgreSQL deployment SQL is included; Tiger Data remain deployment options rather than verified running infrastructure.
Analytics: NumPy and custom robust-statistics code.
Reports: the Google Gemini API with schema-constrained JSON output, validation and a deterministic template fallback.

One change-detection method checks whether the seven-valid-day rolling median's robust z-score is beyond ±1.5 on at least five of the last seven valid days. A separate CUSUM method tracks accumulated deviations. These methods aim to reduce reactions to ordinary variation, but their thresholds still need validation. They do not guarantee that a single unusual day can never produce a flag.

The stored baseline and report pipeline are not fully aligned yet: reports currently calculate their reference from the loaded 28-day comparison history rather than always using the stored initial baseline. Using one consistent baseline across both is remaining integration work.

Gemini's role. The backend calculates the numbers. Gemini receives a structured summary without patient names or dates and writes the narrative. It is instructed not to diagnose, invent measurements or treat a cane drop as a confirmed fall. The backend checks the JSON structure, metric names, numerical claims, clinical wording and consistency with the computed flags. If generation fails or fails validation, the system uses a template report instead. Without a configured Gemini API key, it also falls back to the template.

ElevenLabs' role. ElevenLabs provides the voice for the cane's spoken prompts [CONFIRM and edit: what the prompts say, e.g. asking the user to respond before escalation]. It only affects how prompts sound; detection and escalation logic run in the Pi software.

Demo data. Our longitudinal demo uses eight weeks of clearly labelled synthetic cane-use history. It includes a baseline period, stable activity, a simulated decline and missing-data days. It demonstrates the software's behavior; it is not real patient history or evidence that the system can measure disease progression.

Challenges we ran into

Hardware availability. We didn't get all the parts we planned for (a speaker and an ultrasonic distance sensor), so the cane plays its voice prompts through a phone instead for demonstration purposes.

Distinguishing cane movement from human events. A cane being planted, dropped or set down does not reliably tell us what happened to its user. We kept the event labels cautious and added cancellation and escalation flows rather than claiming confirmed fall detection.
Integration across several components. The repository contains cane software, phone sync, a mobile app, backend analytics and a report pipeline. Having those components in code is not the same as validating a complete physical system. Real-hardware validation and consistent report baselines remain work to finish.
Working with incomplete data. A missing record can mean the cane was off or not synced. The analytics exclude missing or invalid days rather than treating them as zero activity.
Keeping generated reports grounded. We separated numerical calculations from narrative generation and added validation and a template fallback.

Accomplishments that we're proud of

A connected software design spanning cane-side detection and logging, phone sync, an app, backend analytics and reports.
An offline local detection path that does not wait for the backend or Gemini.
Explainable comparisons based on a person's cane-use history, with missing-data handling and configurable change-detection rules.
Clear limits on the claims we make. The system records cane-use patterns and potential events, without presenting them as diagnoses or confirmed human falls.
A repeatable simulated demonstration for cancellation, escalation, activity history and report generation.

What we learned

A cane sensor measures cane movement. Turning that into a useful clinical signal requires validation, not stronger wording.
Local detection and remote notification are different capabilities. An offline detector does not make an internet-based caregiver alert work offline.
Missing data, reference periods and cancellation flows matter as much as the main detection logic.
Simple statistics are easier to inspect and explain, but their thresholds still need testing.
Generated text needs checks against the underlying numbers and a fallback when those checks fail.

What's next for HandyCane

Validate the physical build. Real-hardware bring-up has started: the cane-mounted LIS3DH accelerometer has returned sane live readings on the physical Pi and an on-cane calibration has been saved, and a bench test of the button passed. A full mounted detection run, audible voice output through the paired phone [and ElevenLabs prompt playback], and end-to-end sync from cane to app to backend are not yet verified. Confirm wiring, audio output and end-to-end sync before making hardware-performance claims.
Test safely with people. Start with supervised, staged cane scenarios and usability testing. Any later clinical study or comparison with measures such as the 10-metre walk test would need appropriate oversight and ALS clinic input. We have not validated detection on real human falls.
Finish the integration. Use the stored patient baseline consistently in reports, enable and verify scheduled report generation in a deployed environment, and verify the complete cane-to-app-to-backend flow.
Improve detection with cane-specific data. Evaluate the threshold method before exploring lightweight models. Body-worn sensor data should not be assumed to transfer to a cane.
Explore a second help trigger for situations where the cane is not in hand.
Review privacy and regulation. Assess applicable Canadian and Ontario privacy requirements and the regulatory path before real-world use. The prototype should not be described as compliant or safety-certified without that work.