Voice Order

Voice Order app logged into Burger Boy account.
Admin interface showing menu.
Voice Order ready to take an order.
Creating a new order via voice commands.
Ready to accept payment.
Voice Pay app while creating order.
Voice Pay app ready to make payment.

Inspiration

As a care giver, I have been hyper-sensitive about my contacts in this COVID-19 world. In looking for ways to use technology to improve on processes, I considered my local small businesses. Order taking requires risky face-to-face dwell time that can be limited with the right solution.

What it does

Voice Order implements an automated contactless order taking and payment service using a voice-enabled kiosk and Square services for a catalog of items, order logging, payments, and loyalty rewards. The kiosk runs on an iPad and interfaces with 2 printers - one for the customer to print their order ticket, and one for the kitchen or fulfillment area for prepping the order. There is a companion Voice Pay app that interfaces over bluetooth with the kiosk Voice Order app. The Voice Pay app identifies you to the service, and enables contactless payment via a credit card with Square, or Apple Pay.

The merchant logs into the Voice Order app and can set up their service by identifying their printers and reviewing their menu. They can also review recent orders. When ready, they enter kiosk mode in the app to take orders. A user launches the Voice Pay app and brings their phone near the kiosk to sync. They are then presented with a guided menu of things they can say on the kiosk to place their order. As they speak to the kiosk, it assembles their order and shows the total cost. The Voice Order app automatically applies any earned rewards, such as a free item. When done they can add a tip, and finish their order. The kiosk then prompts to make a payment on their Voice Pay app. When that's done, the kiosk shows confirmation and prints a confirmation slip on the nearby Brother printer. This serves as an order confirmation for the user, an includes their order number, order items, total cost, and loyalty points. The order is also printed on the "kitchen" or backroom printer for preparation or fulfillment.

The system is then ready for the next customer.

The whole system uses the Square service to host the order item catalog, to record orders, take payments, track customer loyalty, etc.

How I built it

I created 2 iOS apps in Swift - Voice Order and Voice Pay. I used the Brother SDK to interface to the 2 printers. The Voice Order app uses iOS voice recognition technology to implement the order taking process. Both apps use bluetooth technology to synchronize information between them. They both use the Square SDK to interface with the payments service. The Voice Order app further interfaces with the Square service over their REST API.

Challenges I ran into

There were numerous challenges in building this service from scratch. First, getting the voice interface working so that one can navigate through a series of hierarchical menus took a lot of experimentation and testing. Second was interfacing to the Brother printers, which took a while to learn, but I had some help from Brother support. Third was implementing the bluetooth sync interface between the two apps. That was very hard to debug any issues. Finally, interfacing to the Square service took a while to figure out, debug, and test. The payment SDK was somewhat easier to integrate, and there was some good example code. But the REST API required a lot of experimentation with the API explorer, trial and error in the code implementation, and proper sequencing of the calls to effect the transactions.

Accomplishments that I'm proud of

I'm very happy with the progress I was able to make in just a few weeks. While it still needs work before deployment, it's working well as a demonstration. There were a lot of pieces that all had to work together to even get to that stage.

What I learned

I learned a great deal about each of the various pieces. How to implement a voice-controlled interface, how to talk to Brother printers, how to sync information over bluetooth, and how to work with the Square service with their SDK and REST API.

What's next for Voice Order

I want to identify a local merchant where I can field test the service. There is still work to do for account creation, merchant definition, menu editing, and some other components before Voice Order is ready for deployment.