GEMINI SIGHT

• 

  The main interface of the app.

What inspired me: I built this because a close family member is visually impaired. I've seen how hard simple things like walking alone or crossing the street can be. When Gemini 3 Flash came out with its powerful high-res vision and fast thinking, I realized: this could really help people like them move around more freely. The hackathon's focus on real-world multimodal AI impact felt perfect—I wanted to create something truly useful. How I built it: I jumped straight into Google AI Studio for quick prototyping. The app auto-starts the camera and mic when opened. Gemini 3 Flash (gemini-3-flash-preview) does the heavy lifting: it analyzes live camera images at high/ultra-high resolution to detect obstacles, read signs, spot traffic lights, describe scenes, and more. Voice commands like “start the app” or “stop the app” control it completely, and it speaks answers back. I used chained prompts so it reasons step-by-step (e.g., see crosswalk → check light → guide crossing safely). The 1M-token context keeps everything consistent even on long walks. I tested everything on my phone and kept tweaking prompts for better speed and accuracy. What I learned: Gemini 3 Flash is incredibly strong at multimodal tasks—high-res vision picks up tiny details I never expected, like distant small text. You just have to tune resolution carefully to avoid slowing things down or draining battery. Agentic workflows are amazing: the model plans and chains actions like a real helper. And for accessibility, voice-only, fast, reliable is everything. Challenges I faced: Real-time camera processing on a phone was tough—lag and battery drain were real issues. Early prompts were either wrong or too chatty, so I spent time making responses short and clear. Voice recognition in noisy places was tricky too. Live multimodal debugging was new for me, but hackathon docs helped. The deadline was tight, but watching it actually guide someone safely made every minute worth it.

Built With

• cloud-services
• es6+
• gemini
• gemini3api
• html5
• react
• typescript