• GIF

  Prototype for the ultimate goal to launch a beta experience to the public, currently at core functionality, phase zero.

• GIF

  App in core phases 0-1, with the ability to detect objects in a live camera feed based on technical requirements.

🌍 Language Lens – Breaking Language Barriers with AI & AR

🚀 About the Project

Language Lens is an innovative AI-powered mobile app that uses real-time image recognition and translation to help users instantly understand the world around them. By simply pointing their phone camera at an object, users can identify it and translate its name into any language, making learning more effortless, immersive, and inclusive.

Inspired by the challenges of language barriers in everyday life, we set out to create a tool that empowers users—whether travelers, students, or multilingual families—to engage with the world in a seamless and intuitive way.

🛠 How We Built It

We developed Language Lens using modern Android technologies, leveraging the power of AI and real-time processing to deliver an intuitive and high-performance experience.

Tech Stack

• Jetpack Compose – For a modern, declarative UI
  
• Kotlin – The backbone of our Android development
  
• CameraX – For real-time image capture and processing
  
• Retrofit – To communicate with the Google Gemini AI API
  
• Google Gemini AI – AI-powered object detection and text translation
  
• Coroutines – For efficient asynchronous API calls
  
• Base64 Encoding – To convert captured images into a format that the AI can process
  
• Logging (Log.e & Timber) – For debugging and performance tracking
  

⚡ Challenges We Faced

🌎 Navigating Time Zones & Team Coordination

As an international team of two project managers, one designer, and one developer, we faced challenges coordinating across three different time zones. Quick decision-making and async communication became essential for us to stay on track.

👩‍💻 Adapting to Team Changes

Initially, our team had an additional developer, but we had to quickly adapt after they had to step away from the project. This left the entire development effort in the hands of a single developer, requiring rapid problem-solving and strategic prioritization to meet the hackathon deadline.

⚙️ Balancing AI, Performance & Real-Time Processing

Integrating AI-powered object detection and translation on a mobile device posed technical challenges, especially when handling real-time image processing and API efficiency. We had to optimize memory usage, caching, and API request handling to ensure smooth performance.

🌟 What We Learned

• Agility matters – We had to quickly adapt our workflow to changing team dynamics and project constraints.
  
• AI integration is powerful but complex – Real-time AI processing requires careful performance tuning and optimization.
  
• Collaboration is key – Even across different time zones, communication and teamwork made this project possible.
  

🎯 What's Next?

This is just the beginning! Future iterations of Language Lens could include:
✅ Offline Translation Mode – Reducing dependency on API calls for faster results
✅ Augmented Reality (AR) Labels – Overlaying translated text on real-world objects
✅ Expanded Language Support – Enhancing accessibility for diverse communities

Through Language Lens, we aim to make the world more connected, inclusive, and accessible—one translation at a time. 🌎✨

Built With

• accompanist-permissions
• camerax
• figma
• google-gemini-api
• jetpack-compose
• meshy
• okhttp