Inspiration
It's been a passion of mine to make AR/VR mainstream and useful, for the last few years. While I never played it myself, the way that Pokemon Go became the killer app for AR inspired me in how simple it made if for a non-technical layman to use an advanced computer vision model to do something in real life is a great example.
Memoars came about as an idea in line with this, how can I make it so easy for persons to get used to the novelty of AR for persons who don't have their own 3D models. I have a background building out an AR plugin for the web, and one of the things that holds persons back is not having their own 3D model to get started. Memoars helps you right over that hurdle by creating a model on the fly that you can share with others, in a fun & easy to share way.
What it does
When creators build a new Memoar, they can upload a memory, an artwork they've worked on, or a future gift they want to share with someone else. This will be overlaid over the surface of a 3D model template, and they can spruce it up by adding clickable links to the model, as well as audio that plays while the AR mode is activated. You can try this out using the Memoar on the homepage.
Once the Memoar is created (it can take less than 2 minutes), I made it my focus to ensure it's as easy as possible to share it with others. They'll be able to print their own QR Code, or share directly to social media/communication apps like WhatsApp, IG, Facebook (yes Facebook, they are in my demo) and others.
On mobile is where the richest of the experience occurs, and I recommend that persons open the experience in Augmented Reality Mode on their mobile.
How I ended up building Memoars
Technology Wise: I relied heavily on a package I'm familiar with called Model-Viewer (from Google), that allows for 3D visualization. They allow you to extend the functionality to do things that are not necessarily in the GLTF spec, like including an audio file that plays, and modifying textures to show videos.
Challenges we ran into
Many, but a few were:
The loading time of the models. To get a nice model with good realistic lighting effects, I ended up taking a larger model as one of the two templates. This caused an issue with setting the materials on this model, because you have to be careful to time it right at the point it loads, to switch the models. This isn't a Bolt issue at all, but we had to switch our entire waiting-till-it-loads approach towards the end, as the model wasn't loading in the live site version.
Bolt issues when it came to the pages rewriting themselves with a completely different base UI when the files got too large :D
Accomplishments that we're proud of
- Honestly, it works better than I thought it would at first, and integrating things like your own voice recording came out great. Tagging the elements in 3D is working well too, even though that can be tricky with the third axis on the plane.
What we learned
One of the big things here was understanding the auth setup of Supabase, and the redirect URLs to get them to work.
Learned about the inner ThreeJS model of the Model Viewer Scene
And not so much learned, but honestly once you have access to the HTML of a scene, so much is possible.
What's next for Memoars
Memoars itself is a proof of concept that I want to use to understand the general public's need for AR solutions. It is to help me guage needs that persons would use and pay for, for my actual product - LixAR. I'm hoping to use these learning to guide me in future development so I build something people actually benefit from.
Log in or sign up for Devpost to join the conversation.