Inspiration
The Interdimensional Portal in Dr. Strange allows people to walk through it to a whole new location. We want to bring a portal to a different world into the palm of our hands (teleportation not yet implemented).
What it does
Strange projects a stylized version of the world through an iPhone. Users can walk through a virtual portal into an altered AR world where they can imagine objects and place them, with 3D models generated by AI.
How we built it
The effects on-screen are created using a neural style transfer model, which runs locally. ARKit and a text-to-3D model are then combined to composite objects on the filtered video.
Challenges we ran into
Compressing the style transfer model into a format that runs at a high framerate on the iPhone was difficult as some of the SDKs provided by Apple were still experimental. We couldn't use a remote server for this either as on-device inference provides better latency. Thus, heavy optimization was required to provide a smoother experience.
Accomplishments that we're proud of
Writing an ARKit app with AI integration in a day was tough but rewarding. We are also proud of our ideation process, coming up with something that is very different from what we typically do.
What we learned
We learned project management skills and balancing the workload between the different skillsets of our members. On hard skills, we refined our knowledge in AI, iOS development, and server-side deployments.
What's next for Strange
We could add support for arbitrary filters based on images and text to make it a true portal to any world. Multiplayer, collaborative worldbuilding is also an exciting idea that could bring worldwide fan bases closer together.
Log in or sign up for Devpost to join the conversation.