Inspiration
Who has time to open an app? Imagine millions of households, where mom is driving back to home after work. She usually calls her favorite place for pizza, or burger joint for a pickup order on the way. Even though online ordering is everywhere, Google My Business points to online partner websites, these vendors have jacked up the price on each item. This is creating more incoming calls, savvy consumers call restaurants directly instead of paying extra. On top of it, labor shortage at restaurants staff is over worked. We are dedicated to democratize access to AI.
What it does
HostBuddy voice bot acts like a Call Center for restaurants and automate the incoming phone call posts the orders to Square Register.
How we built it
We used open-source voice models like Whisper for automatic speech recognition (ASR) to convert voice inputs into text. Afterward, we set up an AI agent and integrated it with a large language model (LLM), embedding the restaurant’s item library and business type etc extracted from POS. Trained the agent with synthetic data (Ordering expressions) generated by GPT-4 to recognize future patterns and improve its responsiveness.
We used the Square REST API to post incoming orders directly to Square Register!!
Challenges we ran into
Accomplishments that we're proud of
Big companies like DoorDash and Uber Eats, haven't built this yet. For example consumer can call for delivery order instead of opening an app. By making it easy, DoorDash will have more delivery orders.
Its not a chatGPT plugin, which is browser based. We built it for voice based, to leverage 100 year old phone lines. Imagine using AirPods, Siri to order your morning Coffee and Bagel, without waiting before the cashier.
What we learned
LLM are still evolving technologies, we eliminated the inaccuracies using RLHF and SFT. I will guarantee you our bot doesn't make up modifiers, price it wrongly from your POS data.
What's next for multi-modal bots for sellers
** Drive Through ** ordering and voice enabled kiosks are next frontiers.
Log in or sign up for Devpost to join the conversation.