Moment Scan

Inspiration

Problem: The camera app is the most commonly used app, but the user experience has always been just the viewfinder - capture and - the end. No major feature has been introduced to this (arguably) most essential app on the smartphone, and in that sense, it's the most boring app you can't live without.

As a first step to building the new geospatially-enhanced camera app: What if you can feel the gravitas of the moment in time that you are capturing - immediately after capturing it - a view of the former present permanently placed for the future.

What if your camera app can let you walk back and visibly time travel back to that frame?

What it does

Moment Scan is a new geospatial photo app platform - that lets you scan each photo moment to crystalize it in geo-space-time, transforming the moment into a future visceral experience... so that the next time you are there, you can "walk into the moment" - and even time-scrub it exactly where it happened.

And also, when you double tap in the middle of the screen, pop up a nifty spotlight search where you can type in what you need point you in the direction of where your keys are

Press the button that looks like cardboard VR to enter the AI-created 360 spherical photo version of this regular picture - we use monoscopic 360 stable diffusion AI outpainting, starting from the selected photo.

Walking about the city, getting lost among the buildings, and not sure if you should keep going that way? Press the button that looks like the bunch of flying rectangles... so you can decide with a little more foresight by seeing photos taken in the same direction.

How we built it

Moment Scan features an integrated UI that elegantly displays the dashboard essentials needed for ARCore GeoSpatial localization.

Essential geo data like lat, lng and horizontal accuracy are succinctly displayed, and, as we twirl around, notice that the capture button features a compass direction as the outer border.

We also have full quaternion rotational tracking from the Geospatial anchor, so hence we display a sphere, so you can track your rotation in realtime. Whee...

And, when we press the Moment button to create our moment, the geospatial camera shutter sound effect, or "plop" that we hear is 3D spatial audio originating from the source.

The Moment Scan Platform is designed to be versatile and useful and aspires to be a high-use daily app as your future camera app of choice.

Challenges we ran into

I had assumed the VPS had a universal trackableId, but it seems that it is randomized each time. Not realizing this led to some strange rabbit holes in debugging.

Accomplishments that we're proud of

I wanted to spend a weekend working on an elegant app from scratch, hacked out using mostly vanilla Unity. I also wanted to build a 3D AR app that has no 3D meshes - yup, the moments are not planes - they are all Unity World Canvas UI's, keeping the entire Unity project Asset folder size less than 20 MB! Pivoting a few times in 24 hours, this app chanced to happen!

It is admittedly a bit rough around the edges, but I truly believe this is a MVP for a game changing AR app with high utility.

What we learned

It seems sub-meter accuracy is possible once we go outside, so this is definitely one of those apps whose demo video should ideally be filmed outside... except it is way too cold at 4 am as I film and wrap up my entry!

I wish I had more time to test out my app in the real world.