FrameSpace

Real time portal of the media lab lobby
Real time portal of the hacker space
real time portal of alternative MIT Reality Hack Hackers
a portal opened up on the hacking floor!

Inspiration

Image generators like Dall-E 2 and Stable Diffusion make imagining incredible and fantastic new realities easier than ever, so we wondered, how can we bring these alternate realities into our own? Imagine being in your everyday space and suddenly interacting with Salvador Dali’s creations, exploring the cosmos, or experiencing first hand what it would be like to have an alien cyborg riding an elephant right next to you. Imagine being stuck designing an interior for a client, when the thought occurs, what if there was a medieval archway instead of a door? By bridging the world generation of Stable Diffusion with the immersive experience of AR, we aim to open new possibilities into how our spaces can inspire us and help us make it a collaborative ARchive. Our project lets us ask ourselves, “What if our current environment belonged to another world?” Imagining other worlds, surrealist environments, and the fantastic supernatural versions of our mundane reality have long been a central goal of novelists, playwrights, and filmmakers, but our experience brings these concepts into a physical space.

What it does

Framespace is an interactive AR experience that lets you explore alternate realities of the space you are present in. Using gestures, we can create portals to other versions of our current reality, as imagined by stable diffusion.

How we built it

We built a custom node.js backend that runs the stable diffusion models on replicate with our custom parameters, to allow real time image generation in under 4 seconds. Our unity project interacts with our node project to

Challenges we ran into

Accomplishments that we're proud of

We were able to create an optimised Stable Diffusion model to run under 3 seconds, and created a performant networking API to return real time generated images to the headset.