KnowWhere

Inspiration

We came to the hackathon with zero prior experience using Google Cloud APIs or generative AI. We wanted to challenge ourselves by building something that combined real-world geolocation data with generative AI.

What it does

KnowWhere is a native mobile app that transforms your surroundings (POIs) into a live guide.

When the app opens:

-it detects the user's current location

-scans for nearby POIs

-ranks and selects the top 5 most popular POIs

-generates descriptions using AI

-speaks these descriptions for the user

How we built it

Frontend: The app was built using React Native with Expo framework for rapid build and deployment to our iPhones. Expo sensors were used to get the search radius from the user's location data and send this through to the backend API.

Backend: Written in Python (Django) as this was a familiar technology. The current coordinates of the user are sent to the backend and calculations are done to show the top 5 most popular places within a given radius. This API integrates with Google Places API, Google Gemini API and ElevenLabs.

Challenges we ran into

Gemini development - generating a short description of a given POI is an intensive task for the Gen AI to complete, therefore we spent a long time optimising the prompt we had given in order to reduce the token usage and improve generation speed.

Expo caching - caching in Expo was more challenging than anticipated, due to constantly changing latitude and longitude values from the user.

Time constraints - due to a 24 hour limit and only a 2 man team, we had to prioritise core functionality and proof of concepts, this is reflected in the final build.

Accomplishments that we're proud of

We have never used any of the AI technologies prior to this hackathon and it was good to get to grips with them and understand how they can add valuable features to an app. We also worked in a small team and had to overcome the challenge that we could not write an insane amount of code within the 24 hours.

What we learned

We successfully built a working end-to-end AI / geographical data application, and collaborated efficiently in a small team. Most importantly, we had fun and enjoyed working with a new tech stack.