Create

Redesign your Living Room
Fight with your Creations is AI Invasion Mode
Transform your Space using Meta's Passthrough API
Original Concept Drawing
Share Your creations directly with the Discord community
Make Literally Anything!

Inspiration

The core inspiration for this project was simple: to create a platform that allows users to instantly generate their own unique 3D environments and assets at runtime—a literal sandbox of possibilities. The goal is to let users effortlessly conjure objects and entire scenes into existence just by describing them, effectively turning imagination into shared, immersive reality.

What it does

This mixed-reality application enables users to generate custom 3D models at runtime and place them in their real room or two virtual scenes. All created models are saved and loaded for persistent environment building, and creations can be photographed and shared directly to a Discord server. While models are rendering and downloading in the background, users can engage in the AI Survival Mode, shooting waves of increasingly fast and numerous enemies. This survival mode features a dynamic fallback, initially using stock models but immediately incorporating the user's unique, downloaded creations as soon as they are ready.

How we built it

Our application was built using the Unity game engine and the Meta OpenXR SDK to leverage the power of mixed reality. The key technology enabling this creativity is the Meshy.ai model generation API, which converts text prompts into 3D models. After exploring several model generation tools, we found Meshy to offer the most reliable service and highest quality output for our needs. We successfully integrated the mixed reality mode and implemented a weekly theme scheduling system. This system uses Firebase to automatically update the application and the associated Discord server with the current weekly theme, providing fresh creative prompts for users.

Challenges we ran into

The greatest technical challenge we faced was securing a reliable model generation API. We dedicated significant time to trialing various solutions, including hosting our own backend and using several third-party services. One promising tool was unfortunately depreciated, and others proved questionable in reliability, or unusable build times. Another ongoing challenge has been maintaining the project through SDK updates, ensuring compatibility with the latest development tools. Also having enough money to ensure the app works is hard to do. Server expenses add up quickly.

Accomplishments that we're proud of

We are proud to have created an application that is engaging, functional across scenes, and provides a seamless mixed-reality experience. It was hard to tie together so many moving parts into one package that is easy for the end user to use.

What we learned

What we've learned is the crucial value of planning out the end goal. While the project has evolved organically from its initial idea—which was beneficial for exploration—the lack of upfront architectural planning led to multiple instances where major refactoring was required because new features demanded radical restructuring of the foundational code.

What's next for Create

Getting the camera capture and upload functionality working in mixed reality mode is a key focus, as this directly relates to the project's core inspiration of creation and sharing. We plan to add a multiplayer mode so people can create worlds together. This will require the implementation of user avatars so that people can interact, and most importantly, take and upload photos together in their collaborative spaces. This will use Meta's Shared Spaces API to accomplish this. We wanted to add this for the competition but ran out of time. We also need to figure out a way to more quickly generate high quality models.