EchoSphere

Inspiration

EchoSphere began with a practical question: how can we help people with visual impairments move more safely outdoors without forcing them to constantly check a screen? Existing assistive tools often solve only part of the problem. Some focus on detecting nearby obstacles, while others focus on destination search or navigation. In real walking situations, however, users need both immediate hazard awareness and continuous route guidance.

We were also motivated by a broader idea of accessibility. While EchoSphere is currently designed for people with visual impairments, we wanted to build it on a multimodal foundation that could later expand into a wider accessibility system for different users and environments.

What it does

EchoSphere is a real-time, voice-first mobility assistant for outdoor walking.

It provides two main modes:

Hazard Detection Mode: Identifies nearby walking risks such as stairs, vehicles, poles, and obstacles from live camera input and delivers spoken guidance.
Navigation Mode: Accepts a destination query, searches for a nearby place, generates a walking route, and guides the user while still prioritizing urgent hazards.

The system also supports voice commands, OCR-based reading from camera images, hazard sensitivity settings, and haptic or directional feedback.

How we built it

We built EchoSphere as a mobile-plus-cloud system.

Frontend: Flutter mobile app.
Backend: FastAPI server deployed on Google Cloud Run.
AI & Data: Uses Gemini for multimodal scene understanding and Google Cloud Vision OCR for text reading.

For pedestrian navigation, since Google’s pedestrian routing is currently limited in Korea, we are temporarily using a local API for the prototype. However, we plan to replace it with the Google Routes API as soon as the service becomes available in the region. Our ultimate goal is to provide a perfectly integrated user experience within the Google ecosystem as soon as local infrastructure allows.

Challenges we ran into

One of the hardest problems was not recognition itself, but prioritization. In a real walking scenario, the system must decide what to say first, when to interrupt, and when to stay quiet—especially balancing hazard alerts and route guidance.

We also faced technical challenges in converting raw route steps into short, understandable spoken instructions and adapting route generation to the Korean pedestrian environment. We realized that the user experience fails if spoken guidance does not clearly communicate direction and movement in real time.

Accomplishments that we're proud of

We are proud that EchoSphere works as an end-to-end multimodal prototype rather than a collection of disconnected features. We successfully connected a real Flutter mobile app to a cloud backend, processed live camera frames to return usable spoken guidance, and built the entire project around actual accessibility needs instead of treating them as an afterthought.

What we learned

We learned that assistive AI is not just about perception accuracy; it is about timing, brevity, clarity, and trust. A useful system must answer when to speak and, more importantly, when not to interrupt the user.

We also learned that practical navigation depends heavily on regional constraints. Most importantly, we learned that accessibility technology should be designed to expand over time by building adaptable interaction patterns.

What's next for EchoSphere

Our next goal is to make EchoSphere more reliable in real-world use. We plan to improve turn-by-turn route guidance to be more action-oriented and make destination naming more specific, such as identifying exact store branches. In the future, we believe this technology can also support indoor spatial exploration and become part of a broader accessibility system that helps reduce friction in everyday life.