A few weeks ago, our team was fortunate enough to compete at the VIA SA Codeathon. Over the course of that competition, we observed several impressive projects, including several that dealt with Augmented Reality. Having never dealt with AR before, our team decided that RowdyHacks 2019 would be the best place to start. At the event, a casual conversation about the frustrations of ordering food from a restaurant only to not receive the dish you expected led to the idea of Smart Menu.

What it does

This program is based primarily in Unity with elements in Xcode for IOS. It enables the user to use their mobile device to scan a picture on a menu (primarily either a QR code or logo) and display various pieces of information on the food served at that particular restaurant. The data is displayed for all items on the menu and can be culled through the user. The information displayed includes the product name and description, nutritional information, and a scalable, interactive 3D model of the dish. If the company or restaurant wishes, they may also display promotional videos of their products in the AR menu as well.

How we built it

We primarily used Unity for this project. Initially, every member of our team focused on learning specific skills that we could combine into the final product. In order to use the IOS system to display our AR environment, we also implemented the code into Xcode.

Challenges we ran into

As we have never worked with Unity or AR before, nearly everything was difficult to begin with. The biggest obstacle was getting around understanding how to use the specific tools that Unity has to offer. They are vastly different from the IDE's that we are experienced in using and they required a steep learning curve. One specific problem that we struggled with was being able to convert data types in C#.

Accomplishments that we're proud of

We are very proud of our ability to learn Unity and AR to the degree necessary to complete our project. We are especially proud of our video implementation of a Pepsi commercial into its' own AR scene as well as our ability to configure a touch screen system to allow for user interaction with our 3D models.

What we learned

This experience helped us learn the Unity IDE as well as Vuforia and gave us a greater knowledge of the functionalities of augmented reality in general.

What's next for Smart Menu

A future development of Smart Menu would be the capability to bring up an entire list of the menu items as the first screen so that the user may more seamlessly navigate to the food items they would like to inspect. One last future application of Smart Menu would be the ability to use similar techniques in order to potentially launch a new project entitled "Smart Label" that could apply to a vastly wider array of retail products and provide companies a new way to reach consumers.

Share this project: