Forward To The Past

Select a historical period from the globe.
Speak with AI agents using your voice.
Immerse yourself in historical sites.

Inspiration

Reading history books can be daunting. What if you could ask any question to any historical figure? Well, now you can!

What it does

Forward To The Past is a VR historical education app for Meta Quest 3 that uses AI-generated interactions with historical characters to make education more accessible to the masses. You can literally speak via the Quest's microphone to communicate with historical figures like Montezuma of the Aztecs, a pharaoh of Ancient Egypt, and a samurai using AI. You can start with simple questions such as "Who are you?" or "What are we looking at?" and then progress to more in-depth queries like "What pivotal roles did you play during the Sengoku period, and what are your favorite military strategies?"

How we built it

Inworld AI was used to create AI NPCs (non-player characters) that users can speak to in virtual reality. We incorporated speech-to-text to interpret the user's questions, and text-to-speech to vocalize the AI-generated NPC responses. We used Unity, C#, and the XR Interaction Toolkit to implement VR interactions with spatial UI. Blockade Labs AI was used to generate the immersive 360 skyboxes. We used Soundraw to even AI-generate the music and sound effects!

Challenges we ran into

Integrating all the AI APIs into Unity and running them on a standalone XR device is as challenging and time-consuming as the debugging process. This unique combination of the virtual reality packages, generative artificial intelligence 3D characters, and voice interaction, was a challenge that involved combining multiple kinds of technologies. Additionally, we need to ensure that the Unity application has a stable network connection and essential permissions to access the microphone.

Accomplishments that we're proud of

In this virtual reality experience, all AI NPCs embody the knowledge, characteristics, and awareness of real historical figures. Engaging in face-to-face conversations with them feels truly magical!

What we learned

Leveraging AI tools to create a more immersive and interactive XR environment.

What's next for Forward To The Past

We aim to transform history learning into a collaborative community experience where individuals can effortlessly share the stories of historical figures they are passionate about using our platform's user-friendly AI, requiring no coding skills. Whether it's a well-known figure or a lesser-known one, users have the opportunity to narrate captivating stories. Through this, our goal is to redefine the traditional 2D approach to history (books and images) into an immersive 3D experience through VR and XR, making it accessible to people of all ages and knowledge levels.