ThirdEye AI – Multimodal Safety Assistant

Inspiration

Many visually impaired, elderly, and cognitively challenged people face difficulty in navigating unfamiliar environments and understanding their surroundings. I wanted to build an AI assistant that could help them feel safer and more independent in daily life using real-time guidance and voice interaction

What it does

ThirdEye is a multimodal AI assistant that helps users: Describe surroundings using camera input Give safe walking guidance and path awareness Answer user questions through voice interaction Support multiple languages including English and Indian languages Provide SOS emergency help and reminders Ask users about their condition and personalize assistance The system is designed especially for visually impaired users who cannot easily type or read screens.

How we built it

The project was built using: Gemini API for multimodal reasoning (text, image, and voice understanding) Gemini AI Studio for prototyping and development Voice interaction features for accessibility Camera-based input to analyze surroundings and provide descriptions The AI is prompted to act as a safety assistant and give concise, helpful guidance.

Challenges we ran into

Designing an interface suitable for visually impaired users Handling voice interaction and multilingual support Managing API limits and prompt optimization Creating a simple but effective prototype within limited hackathon time

Accomplishments that we're proud of

Successfully built a working prototype of a multimodal accessibility assistant Implemented voice-based interaction and personalized user setup Designed a real-world solution with social impact

What we learned

How multimodal AI can solve real-world accessibility problems How to design user-friendly AI systems How to integrate AI tools and APIs in an application workflow

What's next for ThirdEye AI – Multimodal Safety Assistant

In the future, we plan to improve ThirdEye by adding real-time navigation with GPS integration, better object detection, and offline support for low-internet areas. We also aim to expand language support to more regional and international languages and improve voice interaction for faster responses. We would like to integrate wearable devices such as smart glasses for hands-free usage and enhance emergency features like automatic SOS alerts and live location sharing with caregivers. Our long-term goal is to make ThirdEye a reliable daily companion for visually impaired, elderly, and cognitively challenged users, helping them live more independently and safely.

Built With

• accessibility
• gemini-ai-studio
• gemini-api
• javascript
• multimodal-ai
• react-native-(concept)
• typescript
• voice-recognition