PathSense

Inspiration

A white cane is great, but it can’t “see” fast-moving hazards. We wanted a low-cost, phone-only co-pilot that narrates the scene and nudges safe movement—without sending video to the cloud.

What it does

PathSense runs on-device object detection and immediately speaks the object first (“person ahead”, “chair ahead”) and then a navigation cue (“slight left”, “stop”, “caution”).
It streams lightweight event logs to a Fetch.ai Agentverse safety agent that:

1. Detects danger states (stuck, repeated near-collisions, or device silence)
   
2. Sends SMS alerts to a trusted contact
   
3. Archives telemetry to ASI One for privacy-scoped insights that only the caregiver can access

How we built it

• Android (Kotlin, CameraX, Foreground Service, Jetpack Compose)
  
• YOLOv8n → TFLite INT8 (320×320, NMS output [300×6]) via NNAPI/XNNPACK
  
• A planner that maps detections → tokens (STOP / CAUTION / SLIGHT LEFT / RIGHT) with smoothing and dead-zones
  
• TTS + haptics for clear, low-latency cues
  
• Agentverse client (OkHttp) to stream events; Agent triggers SMS and stores logs in ASI One with caregiver-only credentials

Challenges we ran into

• Quantized model scaling & confidence thresholds across devices
  
• Label/index mismatches and coordinate normalization
  
• Reducing “chatter”: debouncing speech, stabilizing cues, handling orientation
  
• Balancing latency vs. battery in a persistent camera service

Accomplishments that we’re proud of

• True on-device guidance with clear object-then-action narration
  
• End-to-end safety oversight via Agentverse + ASI One with strict access control
  
• Robust mobile pipeline that survives screen-off and poor connectivity

What we learned

Edge AI is as much systems work as model work: logging, smoothing, permissions, and UX (the right words at the right time) matter as much as mAP.

What’s next for PathSense

• Stairs/curb depth cues
  
• Finer path planning
  
• Wearable audio integration
  
• Multilingual voices
  
• Caregiver dashboards on ASI One
  
• Structured pilot with blind users to iterate on safety and trust

Built With

• android
• kotlin
• python
• tensorflow
• yolov8