Inspiration

Our journey began with the concept of a robotic safety companion for the visually impaired. However, when evaluating its real-world viability, we realized that relying on specialized hardware heavily limits accessibility. We wanted our solution to be universal. Thus, we pivoted to the most ubiquitous device available—the smartphone—transforming an everyday item into a powerful, real-time safety companion.

What it does

Safe Step is a web-based visual aid application designed for the visually impaired. It utilizes the smartphone's camera to continuously scan the environment. When it detects obstacles (such as people, walls, doors, or stairs), it calculates their proximity and direction, instantly generating a natural AI voice alert to warn the user. It operates hands-free, providing spatial awareness on the go.

How we built it

We adopted a hybrid architecture for maximum performance and cross-platform compatibility. The frontend was built using Flutter Web, allowing instant access without native app installation. The backend is powered by Python and FastAPI, offering a lightweight and high-performance environment for AI processing. For computer vision, we implemented YOLO-World, a Zero-Shot Object Detection model that instantly recognizes custom obstacles via text prompts without requiring massive dataset training. When an image frame is captured, it is sent to the backend where the bounding box's area ratio is calculated to estimate the obstacle's distance and direction. Finally, we integrated the ElevenLabsAPI to synthesize realistic, urgent voice alerts. Notably, all image data is processed purely in RAM as Base64 strings, ensuring Zero-Data Privacy—no visual data is ever stored.

Challenges we ran into

Our biggest initial challenge was hardware dependency. We originally planned to integrate a physical robot as the companion. However, we quickly realized that developing complex robot kinematics within the hackathon timeframe would distract from our core goal: accessible computer vision.

Pivoting to smartphones solved this but introduced new hurdles, specifically navigating iOS's strict HTTPS camera security policies. We overcame this by setting up a secure tunneling network to bridge our deployed cloud frontend with our local GPU-powered backend, allowing for seamless real-time testing.

Accomplishments that we're proud of

We're really proud that we made the camera do more than just name objects—it can actually tell the user the object's direction and distance. Connecting the Flutter app and Python backend to handle live video in real-time was challenging, but figuring it out was our biggest and most rewarding milestone.

What we learned

As this was the first hackathon for all of us, the experience was a crash course in agile teamwork and rapid prototyping. Technically, we gained deep insights into Computer Vision, specifically learning how to extract spatial context from visual data, and how to effectively integrate multiple AI APIs into a cohesive real-time pipeline.

What's next for Safe Step

Our immediate next step is compiling the Flutter codebase into native iOS and Android applications to unlock deeper hardware integration, such as sophisticated haptic feedback and battery-saving Blackout UI modes. We also plan to expand the YOLO-World custom classes to detect a wider array of urban obstacles, such as crosswalks and traffic lights, creating a truly comprehensive mobility aid.