Inspiration

We want blind and low-vision hikers to feel safer and more independent on trails. TrailSense uses a chest-height phone camera to describe nearby hazards through short spoken cues, while adding a fall check-in for emergencies.

What it does

TrailSense analyzes short camera clips for trail dangers such as drop-offs, roots, loose rocks, ice, and people in the path. It gives brief voice warnings and checks overlapping video windows for possible falls. When a fall is suspected, it starts a check-in before escalating to an emergency-contact call.

How we built it

We built a mobile app with React Native and Expo, backed by a FastAPI server. Perceptron Mk1.5 analyzes recent video and returns structured hazard results; OpenCV provides visual fall cues when the fall model is inconclusive. ElevenLabs provides spoken warnings, and Twilio handles emergency-contact calls.

Challenges we ran into

We had to balance useful context against response time, API usage, and log volume. Camera clips are analyzed in windows, so detections can lag behind what is happening. We also worked through mobile camera permissions, network connectivity, and getting consistent video and audio to the backend.

Accomplishments that we're proud of

We connected a live mobile camera flow to structured hazard detection and spoken warnings. We also built a separate fall-check flow that uses both video and audio, with an OpenCV fallback and a check-in before a call is placed.

What we learned

“Live” video analysis still has latency when it relies on sending clips to a server. Short, directional warnings are easier to act on than long descriptions, and safety features need a clear confirmation step to reduce accidental alerts.

What's next for TrailSense

We want to finish reading signs aloud in the app; sign reading is prompted during live analysis, but it is not yet fully connected to the user interface. We also want to measure detection accuracy and latency on varied trail footage, test the experience with blind and low-vision hikers, and improve performance when connectivity is poor.