Languages Go - Language Learning Through Exploration

Pokémon Go meets language learning: 📸 Take photos → AI generates vocabulary cards 🗺️ Real-world pins & collection 🏆 Social leaderboards 🎨 AI-generated cards to collect

Comment

About Languages Go

Inspiration

Over the past 8 months, I've been traveling around Southeast Asia, immersing myself in different cultures and languages. As I wandered through bustling markets in Bangkok, ordered street food in Penang, Malaysia, and explored Singapore, I found myself constantly wishing I could capture and remember the vocabulary I was encountering in real-time.

The most frustrating part? I'd learn a word at a specific restaurant or location, but weeks later I couldn't remember where I had learned it or in what context. I wanted to pin these linguistic discoveries to the actual places where I experienced them - to create a living map of my language learning journey.

Having previously lived in Mexico City for two years, I knew firsthand that the most effective language learning happens through real-world experiences. The Spanish I learned while navigating actual situations stuck with me far better than anything from traditional apps. Now based in Los Angeles, I realized this same challenge exists everywhere - why should language learning be confined to our homes when our cities are full of opportunities to discover new vocabulary?

What it does

Languages Go transforms your city into a language classroom by combining the exploration mechanics of Pokémon Go with AI-powered vocabulary discovery. Users take photos of real-world objects and locations, and our AI instantly generates beautiful vocabulary cards with translations and artwork.

Key features include:

  • AI Vision Processing: Gemma 3 analyzes photos to extract relevant vocabulary words
  • Pin-Based Discovery: Every photo creates a pin on your personal map where you can collect vocabulary cards
  • AI-Generated Cards: Gemini creates beautiful artwork for each vocabulary word, making collection engaging
  • Social Learning: Local leaderboards and activity feeds connect language learners in the same area
  • Real-World Context: Learning vocabulary tied to specific places and experiences for better retention

The app encourages exploration of your environment while building meaningful connections between language, location, and community.

How we built it

This project was built entirely through Bolt.new's One-Shot Challenge, demonstrating the power of comprehensive prompting and AI-assisted development.

Technical Stack:

  • Built on the existing web-camera-kit foundation for camera and geolocation
  • React + TypeScript frontend with Tailwind CSS styling
  • Leaflet maps with OpenStreetMap for real-world pin placement
  • Multiple AI integrations: Gemma 3 for vision processing, DALL-E for card artwork, translation APIs
  • localStorage for data persistence and offline functionality

Relevant Sponsor:

  • Bolt.new for building the app
  • Supabase backend + edge functions (Startup Challenge)
  • Netlify for hosting (Deploy Challenge)
  • Entri/IONOS for domain name

Development Process:

  1. Single Comprehensive Prompt: Crafted a detailed prompt that included camera integration, AI processing, mapping, card collection, and social features
  2. Bolt Generation: Received a complete, working application in one generation
  3. Iterative Refinement: Used Bolt's debugging interface to fix integration issues and polish the user experience

The approach was pragmatic: start with mocked social data for immediate demo capability, then integrate real APIs for the final submission.

Challenges we ran into

AI Integration Complexity: Coordinating multiple AI services (vision analysis, image generation, translation) while maintaining smooth user experience required careful error handling and fallback strategies.

Real-time Performance: Ensuring vocabulary card generation feels instant while processing through multiple AI APIs demanded smart caching and loading state management.

Geolocation Accuracy: Balancing precise location tracking with user privacy and battery life, especially for creating meaningful map pins.

Initial Bolt Generation: The first generation had some integration bugs between the camera functionality and AI processing pipeline, but Bolt's debugging interface made these fixable quickly.

Scope Management: Resisting the urge to build every possible feature and focusing on the core loop that demonstrates the concept effectively.

Accomplishments that we're proud of

Single-Prompt Achievement: Built a complete, multi-featured application with complex AI integrations using just one Bolt.new prompt - demonstrating the power of comprehensive prompting.

Multiple AI Integration: Successfully combined three different AI services (Gemma 3 vision, Gemini image generation, translation) into a seamless user experience.

Real-World Impact: Created something that solves a genuine problem I experienced while traveling - the disconnect between traditional language learning and real-world vocabulary acquisition.

Gamification That Works: The Pokémon Go mechanics feel natural and authentic to the language learning process, not forced or gimmicky.

Technical Polish: Despite being built in one shot, the app includes smooth animations, proper error handling, offline functionality, and professional UI/UX.

What we learned

About Language Learning: Context and real-world connection are everything. Traditional apps fail because they lack the environmental cues and experiential connections that make vocabulary stick.

About AI Integration: Combining multiple AI services creates exponentially more engaging experiences than any single AI feature alone. The magic happens at the intersections.

About Bolt.new: The platform's ability to generate complex, interconnected functionality from a single well-crafted prompt is genuinely impressive. The debugging and iteration capabilities mean you're not stuck with the first generation.

About User Experience: Simple concepts executed well beat complex features executed poorly. The core loop of photo → vocabulary → collection → exploration needed to feel magical before adding social features.

About Prompting: Comprehensive, detailed prompts that include technical requirements, user experience flows, and implementation details yield much better results than iterative feature requests.

What's next for Languages Go

Immediate Roadmap:

  • Real User Testing: Deploy to language learning communities and travel groups for feedback
  • Mobile App: Convert to native mobile using React Native for better camera and GPS integration
  • Enhanced AI: Improve vocabulary extraction accuracy and add context-aware difficulty scaling

Medium-term Vision:

  • Conversation Practice: Enable users to practice conversations using vocabulary they've collected in specific locations
  • Travel Integration: Partner with travel apps to provide location-specific language learning experiences
  • Community Challenges: Group quests and team-based vocabulary collection events
  • Offline Functionality: Full offline mode for travelers in areas with limited connectivity

Long-term Impact:

  • Global Language Exchange: Connect language learners exploring the same cities worldwide
  • Cultural Bridge Building: Facilitate real-world meetings between travelers and locals through shared language learning
  • Educational Partnerships: Work with schools and universities to gamify language immersion programs

The ultimate goal is to transform Languages Go from a vocabulary collection app into a platform that builds genuine human connections through shared exploration and language learning. Whether you're navigating a new country or discovering diverse communities in your own city, Languages Go aims to make language learning as fun as Pokemon Go.

Built With

Share this project:

Updates