Historical Area Explanation

Inspiration We wanted to make history accessible while exploring. Many historical sites lack clear signage, and it’s hard to know what you’re looking at. We built an AR app that identifies nearby historical places and provides context in real time. What it does Historical Area Explanation is an Android AR app that identifies historical places as you point your phone. It: Captures a photo with your camera Uses GPS and compass (azimuth) to determine where you're pointing Finds nearby historical places within a 45° field of view Generates AI descriptions with historical context using Google Gemini Displays a popup with the place name, description, and relevant details Point your phone at a building or landmark to get instant historical information. How we built it Frontend (Android): Android app using Java with camera, location, and sensor APIs Retrofit for HTTP requests to the backend Real-time compass/azimuth tracking using accelerometer and magnetometer sensors Backend (Python/Flask): Flask REST API running on Digital Ocean Google Places API for finding nearby historical locations Custom algorithm that ranks places by heading alignment (85% weight) and distance (15% weight) Google Gemini AI to generate engaging historical descriptions Automatic deployment via cron job that polls GitHub for updates Key Technologies: Google Places API, Google Gemini AI, Flask, Retrofit, Android Sensors Challenges we ran into Azimuth calculation: Getting accurate compass direction required combining accelerometer and magnetometer data with proper rotation matrix calculations Place matching: Determining which building the user is looking at from multiple nearby options - solved with a weighted algorithm considering heading alignment and distance Server deployment: Digital Ocean repository issues required installing pip manually and using --break-system-packages flag Response structure: Ensuring Android app and backend API response formats matched exactly Field of view filtering: Implementing a 45° FOV cone to only show places the user is actually pointing at Accomplishments that we're proud of Built a working AR historical identification system in a hackathon timeframe Created an intelligent ranking algorithm that accurately identifies which place a user is looking at using azimuth and distance Integrated multiple Google APIs (Places + Gemini) seamlessly Implemented automatic deployment pipeline with GitHub integration Generated contextual, engaging historical descriptions using AI that feel natural and informative What we learned How to combine multiple sensor inputs (accelerometer, magnetometer, GPS) for accurate device orientation The importance of API response structure alignment between frontend and backend Working with Google Places API pagination and filtering Deploying Python Flask applications on cloud servers Using environment variables for secure API key management What's next for Historical Area Explanation Image recognition: Use computer vision to match the captured image with Google Street View images for more accurate place identification Offline mode: Cache historical data for popular locations to work without internet Historical timeline: Show how places looked in different eras with historical photos Social features: Allow users to save favorite places and share discoveries Expanded coverage: Add more historical datasets and support for landmarks beyond buildings AR overlay: Display historical information as an AR overlay on the camera view instead of a popup Voice narration: Read descriptions aloud for hands-free exploration

Built With

• android
• java
• python