Find by Fit

Landing Page
Placing size target
Place product

Inspiration

One of the most the powerful features of modern day mobile AR technology is the ability to scan environments to accurately place true-to-scale models in the real-world. So we wanted to utilize this value proposition but flip it on it’s head. So instead of choosing a true-to-scale 3D model and seeing if it fits in the real world we wanted to identify the area in the real world first and use that as a parameter to browse products that would fit in the space.

What it does

The user first places the size target in the real-world and either increases or decreases the size of the target to find the maximum area the product can fit in and then place a product.
The user can then cycle through numerous available products that all fit within the chosen size.
Once a product has been selected, the user can press the buy button which takes them to the associated Product Page on Amazon.
We decided to focus on kitchenware because of the space limitations of this area, the numerous product categories within kitchenware and the frequency of which people purchase new items. However we feel there are many other product categories that this can be applied to

How I built it

We utilized most of the tutorials and some YouTube videos to understand how Sumerian utilized the state machine
We created the toaster models using Maya and Substance Painter
We used the Sumerian API documentation to explore available methods for various entities and classes

Challenges I ran into

The biggest challenge, other than familiarizing ourselves with a new technical environment, was the lack of community driven content to quickly find answers but that’s to be expected given how new Sumerian is. The Slack group was super helpful and the AWS team were very responsive.
The other challenge we didn’t manage to overcome is figuring out how to integrate Sumerian Lex in the mobile app template. Specifically how to grant access to the microphone and integrate that into the Sumerian scene. We granted access to the microphone in the app however we couldn’t figure out how to connect it to the scene. We actually built all the necessary Lex voice commands because originally that’s how we wanted the user to interact with Find By Fit.

Known bugs

During the tutorial section you need to hold the screen instead of tapping because it looks like the touch event happens very quickly which triggers the next screen instantly
The Expand/Reduce buttons are a little sluggish, if the size target doesn’t change instantly give it a few seconds
We didn’t customize the outbound links to each product so it only goes to one product detail page on Amazon
We ran out of time for refactoring -- as we worked in Sumerian more and more we discovered better patterns for scripting and state management, but weren’t able to reimplement many pieces in the time frame.

Accomplishments that I’m proud of

Overall, we’re proud of how it all came together at the end. We really enjoyed how quick it was to test out new builds just by republishing the scene instead of having to rebuild the entire app to the device.
There’s definitely more we could do but we wanted to make sure we delivered an entire application from landing page to onboarding and a clear end point to the experience (arriving on an Amazon product page)

What I learned

It’s always beneficial to see how other products approach familiar problems so it was great to go through the Sumerian tutorials and understand how the IDE was built. We were originally web developers so it was also refreshing to work in JS again.
It was also great to learn how Lex and voice commands were built and to build our own basic version. It’s a shame we couldn’t integrate it into the final app but hopefully we’ll get another chance soon

What’s next for Find By Fit

We didn’t manage to integrate the app into an AWS database that would have been nice to serve the models from which would have given us more flexibility and kept the scene size down
It would be nice to work on adding some more controls to manipulate the model e.g. rotate, dragging the target reticle around the environment
We would also like to add different product categories and additional models
Cleaning the whole app up… a lot!