SamplAR - With Schouse

Inspiration

I come from a Chinese-American background and have frequented Chinese restaurants with my family. Many of these restaurants dine family-style, meaning the everyone orders food to shared together. However, there are many challenges that come with family-style dining on the communication from, and on the coordination front.

Very often, because of the language gap in both communication and being able to read and understand the foods that are orderings, ordering family style tended to have a few people taking control and ordering for everyone, which sort of weakens the value of family-style dining, it's ability to let people bond through working together on the shared goal of deciding food for themselves.

Many of my friends and family had shared this sentiment, causing me to wonder: what if augmented reality, which can make the ordering process both more visual and interactive, can be a solution to this issue?

This lead to me building out SamplAR, and partnering with a restaurant based in Princeton NJ, Schouse, to display foods from their menu.

What it does

SamplAR is my attempt at building a platform that allowed people to order food more collaboratively in a family-style dining environment.

Everyone using the application will be able to see the same thing from their point of view. Users will be able to generate food, and move it around for everyone to see.

This additionally comes with a

1) Real-time display of the price of the food. 2) Ability to toggle on and off the product information for the food.

SamplAR currently has menu items from NJ-based restaurant Schouse - but it's not hard to import in food product and menu items from other foods, and I intend to make the platform generalizable.

How we built it

I built it on Lens Studio, using the Connected Lens Template to allow for data sharing across devices to allow for the syncing of food items and prices across the devices. I scanned the foods using Qlone, an application that uses Photometry to capture 3d visualizations of foods from Schouse, a Chinese Family-Style Restaurant in Princeton, NJ. I stored the models as Remote Assets using the Lens Cloud feature to allow myself to store an expanded amount of food within the lens. I used Javascript scripting to build out all of the additional features.

Challenges we ran into

I ran into many challenges when building this application.

1) Storage Size - there was the limitation of only being able to only store 8 MB into the lens. I overcame this issue by using the Cloud Storage. However, there was the limitation of remote assets having only 25 MB. Originally, each 3d scan was ~ 7MB, which was too much storage to reliably store many scans, because I had tried to use the FBX model. However, I converted it to OBJ models and reduced the size on Blender in order to reduce them to approximately ~2MB each, allowing me to store many more scans into the application.

2) I originally used the hitWorldMesh from the DeviceTracking API to try to place the objects in the world using raycasting, however deviceTracking was disabled to I to having the object spawn right in front of the screen to be able to move it around.

3) Originally, gravity was enabled to simulate the real world environment. However, this made the movement of the items feel difficult to move around, which is why I ended up disabling it for the final product, which seemed to have created a better user experience.

Accomplishments that we're proud of

1) Implementing real-time pricing for the items. 2) Getting the 3d models seem photorealistic for the food models. 3) Positive feedback and insights - from the study I conducted previously.

What we learned

I had conducted 5 studies with 2 participants each, telling them to simulate ordering family-style at a restaurant, and extracted the following insights on how people interact with the application.

1. Participants appreciated the visual allowing users to manage expectations.
2. Participants had a playful experience that allowed them to bond with each other while learning to use the application.
3. Participants didn’t necessarily dislike having less control of the ordering process.
4. Participants tended to fixate on one object at a time when ordering.
5. Participants’ experiences were affected by those of the other participant within their dyad.

These behavioral trends might be useful to inform the future design of SamplAR or any other application built in order to support the collaborative ordering process in the future. Additionally, I believe that working on this project and the very positive reception from the test users demonstrates the immense potential that augmented reality has to impact the restaurant industry - beyond just those that leverage family-style dining!

What's next for SamplAR - With Schouse

Next steps to be done with SamplAR include continuing to develop the product. I would like to integrate more features to support the coordination between two different people, because one complaint that some users had was that it might be difficult at times to coordinate when people are trying to order one thing at a time. I would like to integrate some sort of coloring system. I also would like to make it so there are different ways to visualize the food that is being displayed - as the food product information can be hard to see at times. One test user suggested creating a "grid view" of the menu and also an AR view so that people can use both features and keep track of everything that is there. Although the application is built in order to make it easy for people without much computer programming experience to use Lens Studio and just import the food item and the information, I recognize that there still is the technical barrier to entry of using Lens Studio. I would also like to develop a system - perhaps a simple website that allows restaurants to put in 3d scans of the food product, and the description, and have it automatically be imported to SamplAR. It would also be interesting to partner with restaurants to test a version of their application to better understand the challenges that the restaurants face when using the application.

Built With

• connected-lens
• javascript
• lens-cloud
• lens-studio
• qlone
• snapchat