Inspiration
We were inspired to create Pathfinder after learning how overwhelming indoor spaces can be for blind and visually impaired people—especially in places like schools, airports, or stores where GPS doesn’t help. In a conversation we had with Marci Carpenter, President of the National Federation of the Blind of Washington, she shared how many technologies are built without input from the blind community, and how disorienting unfamiliar buildings such as airports can be. That insight helped shape our approach. Additionally, we had a personal connection—one of our team members has a family friend who is blind, which made this issue feel even more real. Pathfinder is our attempt to reimagine indoor navigation through accessible, voice-driven technology.
What it does
Pathfinder is a prototype mobile app that uses the user's phone camera to help blind and visually impaired users navigate indoor spaces by providing audio cues, voice interaction capabilities, and AI object and text detection. It uses a lanyard that hangs the phone on the users chest, allowing the camera to see while still being hands-free. It alerts the user of obstacles, doors, and signs, and has a completely voice-command driven interface that allows the user to provide questions or commands like “Where’s the exit?” or “Repeat directions.” It demonstrates how AI detection and spatial feedback could one day offer hands-free, step-by-step navigation—making unfamiliar spaces feel accessible and safe for all.
How we built it
We built Pathfinder using React Native, taking advantage of Cursor AI's cutting-edge rapid development tools. Voice command functionality was using React Native's built-in speech recognition and text-to-speech APIs. For object and text detection, we used Gemini to generate realistic detection outputs and simulate how the app could identify obstacles and read signage in real time. We additionally used a simple lanyard with the phone attached, hanging from the users neck, to keep the camera upright.
Challenges we ran into
One challenge was balancing our high ambitions with the scope of a prototype. We had to simulate complex features like object detection and AR for demonstration purposes, while still building something that is functional and meaningful. Another challenge was ensuring the functionality was based upon the real needs of the target audience—not assumptions. After all, on our own, we had zero experience being part of the blind community. To ensure that we were designing with empathy, we reached out to a leader in the blind community for feedback and adjusted our approach based on their insight.
Accomplishments that we're proud of
We are proud of the work we put in to ensure that Pathfinder is the best, most effective solution to our chosen problem. For example, we are proud of the lengths we took to incorporate community input, especially from the target audience. We reached out both to a personal connection we had who was blind, and to a prominent leader in the blind community; they both helped us understand how we could be even _ better _ than previous attempts at solving the issue.
What we learned
We learned how important it is to include input from people with lived experience when designing accessibility tools. We also learned that prototyping isn’t just about the code—it’s about storytelling, empathy, and clarity of purpose and scope. Finally, we got valuable experience with emerging tools like Gemini's API for quick AI detection and response generation, and Cursor AI which helps to vastly accelerate the prototyping process.
What's next for Pathfinder
Our next goal is to fully implement real-time object and text detection using TensorFlow Lite and Google ML Kit, along with AR-based spatial anchoring to estimate steps and directions. In the long term, we envision Pathfinder running on wearable devices like Meta's smart glasses, enabling hands-free indoor navigation that’s seamlessly integrated into the user’s world.
Log in or sign up for Devpost to join the conversation.