HeySalad ® AI HOST

Project name

HeySalad® AI Host

Tagline / one-line summary

An AI host for food businesses that turns missed calls into safe, visible, reviewable bookings and orders.

Inspiration

Restaurants lose demand at the exact moment they are busiest. The phone rings while the team is cooking, serving, or helping customers already in the room. A missed call can mean a lost booking, an abandoned takeaway order, or a customer left without a safe answer.

We built HeySalad® AI Host to give food businesses a helpful first point of contact without taking control away from people. The aim is not to automate hospitality out of the picture; it is to capture the request, work from information the business has approved, and make the outcome clear for the team to review.

What it does

HeySalad® AI Host, represented by Sally, answers customer conversations using the restaurant's approved menu, prices, opening hours, policies, and escalation rules. It can:

capture and confirm a booking, then show the call and structured booking in the operator dashboard;
create an itemised takeaway quote from the approved catalogue and keep the total visible before checkout;
keep calls, transcripts, bookings, orders, and escalations in one operational record;
refuse unsafe requests rather than guessing—for example, an allergy guarantee is escalated to a person with no order or booking created; and
expose the same API-backed event model to future client surfaces, including the HeySalad Chrome extension and a private fridge-camera prototype.

The camera prototype captures a private inventory observation and sends it to the HeySalad platform through an authenticated Cloudflare device service. Observations are deliberately confidence-scored and require human approval before any replenishment suggestion; this demo does not make autonomous purchases.

How we built it

The web product is a Next.js and TypeScript application with a governed business-data model: menus, policies, agent behaviour, calls, bookings, orders, and payments are all visible to the operator. The AI host uses that approved source of truth rather than improvising prices or policy.

For real-time voice, we combined telephony and speech services with a Cloudflare Worker-based voice layer. The device path is Cloudflare-first: an ESP32-S3 camera uses per-device authentication to send events and private images to a Cloudflare Worker, which stores private media separately from the application and emits platform events. The mobile app and Chrome extension consume platform APIs—they do not communicate directly with firmware.

We used end-to-end tests to prove the important outcomes: a confirmed booking for four, a correctly priced £20 demo order, a completed demo payment, and a severe-allergen escalation that produces zero unsafe side effects.

Challenges we ran into

Trust and safety: an AI host must know when it should not act. We designed escalation paths for high-risk food-safety questions and kept a human in control.
Reliable real-time voice: browser, telephony, transcription, and text-to-speech latency all affect the feeling of a conversation. We kept the voice path server-side and instrumented it rather than putting provider secrets on devices.
Keeping one source of truth: menus and policies need to be updated once and then used consistently across calls, the dashboard, mobile, extension, and device workflows.
Edge-device constraints: camera orientation, Wi-Fi provisioning, private image retention, and limited hardware resources all matter before a fridge camera can become a dependable product feature.
Cost discipline: we used Cloudflare Workers, R2, D1, Queues, Durable Objects, and Workers AI for the device/event path so high-frequency edge traffic does not depend on Vercel functions or expose expensive third-party credentials.

Accomplishments that we're proud of

Built one coherent AI-host flow rather than a disconnected feature demo.
Verified a Friday 7 p.m. booking for four people created by the AI host.
Verified a Pad Thai (£13) plus Tom Yum Soup (£7) flow with a £20 quote, quote verification, and demo-payment confirmation.
Proved the safety path: a severe peanut-allergy question was escalated with zero orders and zero bookings created.
Kept the operator in control with visible transcripts, structured records, and human escalation.
Connected the first ESP32-S3 fridge-camera prototype to a private, authenticated Cloudflare device pipeline.

What we learned

The useful unit of AI is not a chat response—it is a trustworthy operational outcome. Restaurants need an agent that can make the routine path faster while leaving uncertainty, safety, and irreversible decisions with a person.

We also learned that the platform boundary matters. A phone agent, Chrome extension, mobile app, and fridge camera can feel like one product only when they use the same event and approval model. Keeping device credentials narrow, keeping images private by default, and routing third-party integrations through the server makes that possible.

What's next for HeySalad® AI Host

Turn the Chrome extension into a lightweight operator companion for reviewing calls, quotes, stock observations, and approval requests while working in the browser.
Add mobile notifications and approval controls for bookings, exceptions, and inventory suggestions.
Complete Wi-Fi and BLE provisioning for the fridge camera, then improve image rotation and capture quality.
Add confidence thresholds, retention controls, and a human approval queue for inventory observations and replenishment suggestions.
Explore conversational device interactions—listen, capture, ask, and speak back—without placing API secrets on the ESP32-S3.
Keep autonomous purchasing out of scope until inventory accuracy, retailer integrations, consent, and approval controls have been proven.