NavSense

Navigation for everyone. Turn directions into haptic cues with smart gloves.

Comment

Inspiration

Audio directions occupy hearing in busy streets, and screens pull eyes away from the path, which adds risk for people who rely on those senses every moment outside. Haptics carry spatial cues through touch, so guidance can land even when sound is loud or vision is busy, which supports safer wayfinding for blind or low-vision people and Deaf or hard-of-hearing people. The goal is practical access to route information through a sense that is open and available in motion.

What it does

NavSense pairs two smart gloves to a mobile app and sends turn cues as distinct vibration patterns that map to left, right, forward, pause, and recalculation signals . Each glove carries an ESP32 that speaks Bluetooth Low Energy to the app, which keeps messages small and responsive while preserving battery life in short-burst transmissions. The app pulls a route from the Google Maps Directions API, parses steps, and assigns a clear haptic pattern to each maneuver with consistent timing windows for advance notice and execution. A voice agent handles start, destination, travel mode, rerouting, and quick questions, using ElevenLabs for natural speech and a Gemini model for robust intent and dialogue control. The gloves are rechargeable over USB‑C and include LEDs for pairing, battery status, and quick error signals that mirror haptic states for cross-sensory clarity. We also integrated machine learning models that can detect surrounding noises like sirens. If detected, the gloves will alert users of potential danger and prompt them to pause before continuing their route.

Who does this help

People who are blind or low‑vision can keep hearing and mobility tools active while receiving turn cues through touch, which removes the need to split attention between a screen and the street. People who are Deaf or hard‑of‑hearing can move without relying on audio prompts, since navigation arrives as tactile instructions that work in quiet and in noise equally well . Runners, cyclists, and commuters gain a heads‑up, hands‑free option that reduces screen checks and helps keep focus on traffic and terrain in motion . This supports the spirit of UN SDGs including SDG 10 Reduced Inequalities, SDG 11 Sustainable Cities and Communities, SDG 3 Good Health and Well‑being, and SDG 9 Industry, Innovation and Infrastructure by improving access, safety, and inclusive mobility tools.

How does it help

Haptics free eyes and ears by moving guidance into the hands, which preserves situational awareness in crowds, in traffic, and in low‑light or noisy conditions . Each pattern is short and unambiguous, like a left‑hand buzz for left turns, so cues land fast and do not overwhelm . NavSense works alongside canes, guide dogs, and safety apps by adding layered information through touch rather than replacing existing aids

How we built it

The iOS app and the voice agent collect the start, destination, and mode by voice, then sends a route request to a server that calls the Google Maps API for detailed legs and steps in JSON form . The app, connected by BLE, can connect to both devices simultaneously and send ASCII to each individual gloves based on the required haptic feedback with small vibration motors connected to the ESP-32. These vibration patterns are pre-coded into the device's hardware (ESP-32). We also custom 3D printed a box to fit on the device and house the components. The LEDs are connected to the pinout of the motors to illuminate when a signal is sent. Additionally, we used a BMS system for our 18650 battery to make it easily rechargeable with USB-C, so the battery can power the device as needed. We used ElevenLabs to provide natural voice for prompts and summaries, while a Gemini model resolves intents, follow‑ups, reroutes, and question answering in a conversational loop over webhooks.

Challenges we ran into

Like any hackathon project, NavSense came with its fair share of roadblocks. Since our app depended on Bluetooth and navigation modules that required Xcode, having only two Macs made development tricky. We had to constantly switch devices and pass around small code changes just to test builds and verify functionality, which slowed us down. On the hardware side, one of our ESP32 boards ran into a bootloader issue during glove assembly, forcing us to reflash and rework parts of the code before we could get it stable again. Time management was another challenge; several team members were balancing midterms and other commitments, so coordinating availability and keeping momentum wasn’t always easy. Despite these setbacks, we adapted quickly, divided responsibilities, and kept pushing forward, which made the final result even more rewarding.

Accomplishments that we're proud of

Our biggest accomplishment is building an app that promotes inclusivity and accessibility, a tool designed to support people with disabilities and help them navigate daily life more safely. This mission was central to our project, and seeing it come to life in a working prototype was incredibly meaningful for our team. On the technical side, we’re proud to have created our first-ever iOS app, taking an idea from scratch all the way to real device testing. We also developed smart gloves that use both LEDs and custom vibration patterns, tuned to deliver clear and reliable navigation cues. For routing, we integrated Google Maps so users can smoothly move from their starting point to their destination with step-by-step guidance. Finally, we went beyond traditional maps by adding voice support powered by ElevenLabs and a Gemini model, enabling users to converse with the system and control navigation at their own pace. Together, these accomplishments showcase not just what we built, but the impact it can have in making navigation safer and more inclusive.

What we learned

This project pushed us to grow in both technical and collaborative ways. We learned how to build vibration-enabled smart gloves by bringing hardware and software together, making sure the signals were reliable and meaningful for navigation. On the app side, we gained experience designing a React Native iOS app with a clean UI and integrating APIs like Google Maps for real-time routing. Implementing an AI agent was another big milestone, teaching us how to use models for natural voice guidance and interactive navigation. Just as importantly, we learned how to collaborate as a team under pressure, dividing tasks, syncing code, and problem-solving together to meet a tight hackathon deadline while many of us were also balancing midterms. In the end, we not only improved our technical skills but also developed a stronger sense of how teamwork and communication are just as critical as code in getting a project across the finish line.

What's next for NavSense

We see NavSense as just the beginning of what wearable-first navigation can become. Our next step is to enhance the haptic smart gloves with a broader range of vibration signals. We want to build a “friends and safety contacts” feature, where trusted contacts can view the user’s live location and route through a web dashboard. This way, if a user is unexpectedly stopped for an extended time, their contacts can step in to check on them. Beyond software, we aim to refine the glove design itself, making it smaller, lighter, and more comfortable to wear for longer periods. Finally, we’d love to expand NavSense beyond accessibility into broader use cases, like runners, cyclists, or anyone who wants a safe, screen-free navigation experience.

Built With

Share this project:

Updates

Submission history