Inspiration

Return to Ithaca is an AR video game which uses cutting edge AR technology, non-scripted AI-powered NPCs, and GPS to recreate the Odyssey in the real world. It uses a bounded real-world map showcasing real landmarks (e.g. college campus buildings) allowing interactive tours and encouraging walking, taking the boredom out of tours and exercise.

What it does

You play as Odysseus, and just like your mythical counterpart, you start from Troy but your journey back to Ithaca is less than comfortable. Troy is automatically set to your starting landmark, but which landmark represents Ithaca is a mystery. You can try your luck by visiting the locations inside the map, and you may come across Ithaca... or any of the other Eldritch horrors we placed inside. However, for the major ones you have to go through one of our monsters.

If you have the patience or the mental fortitude, you will find it useful if you land in the territory of the Sphinx. She is bored and finds it very entertaining to beat around the bush, riddle you, and annoy you so you do not get the clue. However, solve enough of her riddles, and you may get the clue.

If that does not work for you, you can step out and try to find the Cyclops, our hardest boss which will physically fight you. It also has the best clue, with the catch being that you may die brutally by his hand if you lose.

You may also be hijacked by Cyber-Medusa. You will have 3 seconds to look away, as after that, eye contact with your screen will freeze you. You may leave, but if you manage to look at your screen without immediate eye contact, you may find yourself another clue, giving a silver lining to the whole instant death thing.

How we built it

We built the app in Unity with AR Foundation and ARKit for iPhone. We adapted an existing Mapbox story engine by graeburn into an exploration game inspired by The Odyssey. Players scout their surroundings through the rear camera; encounter mythological monsters which we used open-source assets for, redid their animation, and added some minor VFX to start with; and collect clues about Ithaca in a journal that saves their discoveries between sessions.

We designed each encounter around a different physical action, creating an all-immersive experience that players have to act out in real life. To fight the Cyclops, players tilt their phone up to raise a shield, time a block to the gold flash, then tap to counterattack. We reward precise parrying, allowing an extra hit to be dealt on a 'perfect block'. To survive Medusa, they have to look away or close their eyes while the iPhone's front-facing tracking checks their gaze. Unlike the Cyclops or the Sphinx, the Medusa does not have an interactive object on the map; this is CyberMedusa, who hijacks the user's phone, gives them a jump scare, and forces them to tap the phone without looking at the phone. For the Sphinx, we used ElevenLabs to make a custom voice that resembles the Sphinx, intentionally snarky and annoying. We also used Gemini API to generate riddles, match the answer, and keep track of the number of riddles answered. We also built an Oracle chat interface and a Python proxy for Gemini so players can ask about the rules and receive cryptic messages from her.

Challenges we ran into

Medusa was our trickiest encounter because she changes how the app uses the camera halfway through the game. We had to switch from rear-camera AR to front-camera face tracking, then restore exploration when the encounter ended. We also had to get an older Unity and Mapbox project to build and launch with a modern Xcode and iOS toolchain. We also tried to implement the project for an Android build at first, but none of the Android devices in our team had ARCore support, which was a required component.

Accomplishments that we're proud of

We built a fully functioning MVP for our idea despite many challenges, including making the successful fork work through the version control and device availability issues only at midnight. We could work together as a team in complete synergy, and although we may have liked to add a few more features, we certainly achieved what we set out to. Only one of our teammates had experience with Unity and we're proud that we were able to delegate tasks perfectly to match each person's strengths. We're also proud of our usage of Generative AI, where we used the agent as a coding assistant rather than the one generating the idea. Our idea was entirely our own, our research for existing work was authentic, and we spent a total of 5 hours just to get every single prompt detail right before the AI started writing the code. We ran extensive tests, reviewed all features manually, and stress tested the applications where possible within 24 hours to produce an MVP.

What we learned

We confirmed that shooting for the moon helps. Our original ideas had a lot more, but falling short still landed us with a solid fully functioning project. We learned new technologies like Unity and expanded our creativity to include new uses of familiar technology such as Gemini API in new contexts. We also learned, in great depth, the pain of dealing with legacy systems and a lack of backwards compatibility. For example, the project we built upon used Unity 2019, and even though we had a latest Android phone, it is so new Google has yet to receive a certification for ARCore. We also learned how to navigate teamwork and collaboration in an increasingly AI-powered development environment.

What's next for Return to Ithaca - the ultimate AR odyssey

We have tons of new features to add, from new monsters to entirely new game mechanisms. We had plans to add a fourth monster Hydra, which multiplies when killed and has a poisoning effect. Right now, we only support 2D mapping, but in future we plan to add a height attribute with which we can place places of interest on higher floors. Right now, it's a fixed ultra-specific map. We want to add custom maps, a game master mode where a human sets the game up, and randomization of obstacles rather than fixed positions. Along with even new monsters, new mechanics of navigating through monsters, and potentially reward systems where allies are also placed in places of interest, the sky is the limit!

Built With

Share this project:

Updates

Submission history