RealityShft

Inspiration

RealityShft is meant to replicate the phenomenon of fantasizing the world around you. Sometimes, the monotony of the real world leads to bursts of creativity through the synthesis of your surroundings. The initial thought was what if we could see the real world through that lens digitally? Given the constraints, a simplified panoramic version was made.

What it does

RealityShft is a simple desktop application which locationally daydreams. You select a location on a map, and provided there is data for it, it will generate a panoramic view of that location as if from a slightly different dimension using stable diffusion.

How we built it

The application consists of a couple of different components interfacing with one another. The general data flow consists of getting coordinates from the map, retrieving street images for the coordinates, stitching the images together, and then feeding them to an external AI. This is all done in python using google and dream studio APIs.

Challenges we ran into

There was a significant learning curve and consideration of time constraints. Initially, a web app was intended but forsaken in favor of producing a working prototype. Additionally, better API's or in house solutions would be needed to speed up the process time for an idealistic application (a significant amount of time is taken stitching together images since Google's API refuses to give full panoramic images).

Accomplishments that we're proud of

Despite repeat backtracking and starting very late, a working prototype was still completed, and we learned much about the hackathon experience.

What we learned

Though not used, we picked up some react, webdev, and image processing knowledge along the way.

What's next for RealityShft

Due to the lack of hardware capabilities, it was impossible to run stable diffusion on a local machine. But once a more efficient and non nauseating form is developed, it would be nice to return to this for more real time AR experiences long term, with a greater focus on insertion rather than scene translation. Short term, improving the UI and API usage and adding a proper panoramic viewer/diffuse would be ideal.

Built With

• dream-studio-api
• google-maps-apis
• python
• tkinter