God Mode - Create and Destroy

Inspiration

3D asset generation combined with Rage rooms as a way to blow some steam off.

What it does

You can create anything you want by requesting it via voice, then destroy your creations with a bat, and then watch your creations explode.

How we built it

We built a voice to text to image to 3D model pipeline for the object creation side [Oscar, Gustavo, and Patricio] 3D art [Jesus] The VR bat physics and interactions [Braulio] We used Unity, Meta’s Presence Platform SDK, and wav to vec for voice recognition, stable diffusion for image generation, custom flavor for marching cubes 3D model generation.

Challenges we ran into

• It was unclear whether a voice powered pipeline to create 3D models was possible, but we managed to make a prototype in a short period of time.
• Find a Unity compatible format to present the generated 3D models in VR.
• The usual last minute merging code problems that happens when working as a team on a tight deadline.

Accomplishments that we're proud of

Built a full interactable pipeline that can generate a 3D model from a voice input, and then interact with it with realistic 3D physics in VR.

What we learned

• That is possible to do a full AI powered pipeline to generate 3D models and use those models in VR.
• To limit scope of the project

What's next for God Mode - Create and Destroy

Come to work at Founders Inc to develop it further.

Built With

• and-wav-to-vec-for-voice-recognition
• marching-cubes-3d-model-gen
• meta-presence-platform-sdk
• meta?s-presence-platform-sdk
• stable-diffusion
• stable-diffusion-for-image-generation
• unity
• wav-to-vec