Wander

AR Integration

Inspiration 💡

"An intoxication comes over the man who walks long and aimlessly through the streets. With each step, the walk takes on greater momentum [...] ever more irresistible the magnetism of the next street corner, of a mass of distant foliage, of a street name." - Walter Benjamin

It is our belief that the world needs more wanderers. Many people look over to distant lands when they think of exploration, but overlook the wealth of culture, history, and beauty in their own cities. We wanted to make it easier, more fun, and potentially even more healthy to explore and take mental health walks to wander your local neighbourhood or city by creating an AI-assisted tour guide and running coach, so get active and get exploring!

What it does 🗺️

Wander is a mobile-first web app that harnesses Cohere AI to automatically generate personalized guided workouts. Your AI coach will motivate you to keep up the pace, while also giving you interesting facts about the buildings, landmarks, and scenery as you pass by. And if you want to take a break from the running, you can learn even more about each point of interest (POI) through our augmented reality experiences. Just open your camera from the Wander app, scan a QR code located somewhere near the POI, and start discovering a virtual world full of 3D artifacts. With Wander, your city can be like a gym and a museum at the same time!

How we built it 🛠️

Mockup/prototype was designed using Figma, with some help from plugins like Tabler Icons and Mapsicle. Our database service is Firebase, and the backend API is built with Node.js and Express. AR experiences were developed using echo3D, Vectary, and Unity. Other frontend components were developed with React. Our app also makes use of the following services:

Cohere API (for generating a script for each audio guide)
Microsoft Azure Text-to-speech (for synthesizing audio)
Wikipedia API (for finding nearby POIs)
Mapbox Directions API (for step-by-step walking/running directions)

Challenges we ran into 🚧

One of our biggest struggles was integrating each group member's work into one cohesive unit with different technologies. In order to maximize our productivity, each member specialized in one specific area and bring it all together in the end. However, we ended up with very little time to connect all the pieces, we ran into roadblocks with connecting the frontend to other API services.

And of course, like in all hackathons, time is a very limited resource, and we found ourselves wishing we had several more days to work on this. Our vision for Wander is grand and we were hoping to implement a lot of functionality, but with 4 people and 36 hours, we managed to complete a decent amount of our end goals.

Accomplishments that we're proud of 🏆

We're proud that we were able to flesh out our vision of exploration and adventure, and have made the first steps towards building a usable, impactful application. We're also proud of our ability to work as a team, despite being strangers to each other just a couple days ago.

What we learned 📖

All of us were new to the Cohere AI platform as well as the echo3D software, and it took some time to wrap our heads around it and learn how to write prompts that gave us the results we wanted. But after lots of trial and error, we (mostly) made it work and have a better grasp on how to "trick" it into giving us great responses.

This was Daniel's first time designing, prototyping, and then implementing a multi-page application from scratch. He's a pro at Figma's keyboard shortcuts now!

This was also Matthew's and Daniel's first hackathon. We have learned A TON from this experience, both good and bad (one could say we have explored the depths of sleep deprivation and the heights of anxiety...), but mostly good! We found that in order to go from idea to app in such a short time requires a lot of quick thinking, communication, and cooperation.

What's next for Wander 🏃

Our app still has ways to go before people can start using it, and we plan to continue development in the future (if time permits) so that our frontend application looks and feels like how we envisioned it in the prototype. We will also consider transitioning the React code into React Native, so we can build a native application that can better integrate with your devices. This will also allow us to publish Wander on the App Store or Google Play Store.