Apollo GPT

Inspiration

Apollo GPT for us began as a tool for interactive Research and Cyber Security functionality for our Virtual Reality platforms.

What Apollo GPT end up becoming was a fully interactive AI model that became far more than a simple research tool or a security model in our VR environment!

What it does

Apollo GPT is an AI-driven VR platform that creates personalized, immersive experiences by integrating ChatGPT API, Azure TTS, and the Unreal Engine. It adapts to your needs in real-time, revolutionizing the way you learn, work, and socialize. Key features include:

Immersive learning and training with real-time feedback, tailored to individual needs.
Seamless remote collaboration, fostering productivity and innovation across global teams.
Access to virtual healthcare information and directions to key services.
Personalized entertainment experiences, such as interactive storytelling and virtual events.
Enhanced socialization through realistic virtual environments, reducing isolation.
Innovative business solutions, including virtual showrooms and employee training.

How we built it:

Voice SDK by Meta: Captures and processes the microphone data. Azure Cognitive Services - STT: Converts the captured speech to text. OpenAI Chat GPT 3.5/4.0: Processes the text input and generates a relevant response. Azure Cognitive Services - TTS: Converts the generated response text to speech. MetaHuman SDK: Generates procedural animations based on the generated speech. MetaHuman: Converts Face Mesh data into 3D NPC model files. Quixel Bridge: Handles rendering and integration of the 3D models in Unreal Engine 5. UE5 (Unreal Engine 5): Renders the scene with the 3D models and animations, displaying the final output to the user.

Challenges we ran into:

The processing power needed to integrate the functionality of our demos were unable to properly combine each tool together to provide the results we wanted.