Text Tavern - Order food through SMS.

Imagine you are in a busy bar or fast casual restaurant, instead of waiting in line to order food/drinks, consumers can text their drink order to restaurant SMS bot.

Comment

Inspiration

No one enjoys waiting in line, whether it's at a busy bar on Friday or a fast-casual restaurant. While many apps have been designed for ordering drinks and opening tabs, consumers often go unused.

Texting is second nature to most consumers. Instead of creating yet another app or relying on platforms like WhatsApp, we've pioneered a unique approach. Restaurants can now have branded numbers like "1-800-TXT-TAVERN", echoing familiar names like "800-FLOWERS".

Ordering is as straightforward as texting "One cheeseburger with relish and bacon."

What it does

Most of the SMS integrations are super dumb, they mainly provide confirmation link and reminder. Consumer can't text back like you do on ChatGPT and engage with restaurant or bar for great on-premise experience.

How we built it

Utilized chatGPT to generate synthetic data. The prompt requested sample interactions with online restaurant menus provided in pdf and jpg formats. This process yielded several thousand simulated conversations encompassing food orders, inquiries about dishes, and item modifications, capturing a range of contexts.

Used Google Vertex AI to train a BERT model on this data, honing in on understanding user intents, extracting order details, and handling contextual dialogues.

Used Plivo as SMS provider: Set up a Plivo SMS webhook and integrate the trained BERT model using Vertex AI prediction endpoints to process incoming SMS and respond appropriately.

Context Management: Implement a session-based mechanism,using phone numbers as session IDs, to retain context between SMS interactions, ensuring the bot can manage multi-turn conversations effectively.

Challenges we ran into

Well known problem with LLMs is hallucinations, make sure chatbot does not invent anything new for example make up modifier, that doesn't exist. And able in all cases to activate the correct function, and deliver accurate order. As you can see in the receipts.

Accomplishments that we're proud of

SMS integrations always been dummer, we are proud that we can bring human like conversations to SMS, while conversations are still grounded. For example consumers can't order something that doesn't exists in the Square merchant inventory.

What we learned

RLHF is still emerging fine tuning technique.

What's next for Text Tavern - Order food & drinks through SMS.

Take the conversations beyond, sms, phone lines to drive through automation, bring the AI efficiency to eliminate tedious tasks for our restaurant community.

Built With

Share this project:

Updates