TappyTabs

Inspiration

Ordering as a group takes FOREVER. We've all been there: someone asks “are we sharing?” three times, nobody knows what they want, and the friend with a peanut allergy has to remind the table again. We wanted the table to sort out the order before anyone even picks up a menu.

TappyTabs makes that process a lot easier. Each person's tastes and dietary restrictions have a place in the decision, so the group can get to an order faster without leaving someone out.

What it does

TappyTabs turns a group order into a short show of personal taste agents. Everyone taps their phone against the restaurant's phone to join the table, and the AI agents help decide what to order.

Build a taste profile once. Each diner enters their allergies, diet, spice tolerance, food preferences, and price preferences. They pick a character for their agent: tomato, mushroom, bok choy, garlic, or chef cat. Claude writes a taste summary that the diner can edit before saving.
Tap to join. The host sets the party size. Each diner holds their iPhone near the restaurant iPhone. Ultra-wideband ranging detects when the phones are close, a buzz confirms it, and they're at the table. There's also a QR code option.
Watch the agents talk. Each diner has an agent that knows their tastes and dietary needs. The agents propose personal dishes and shared plates, then their avatars talk through the choices in speech bubbles on the restaurant screen.
Pick, then confirm together. Each phone shows its agent's top pick and alternatives. Diners can change their picks and adjust the group order before everyone confirms. In auto mode, the app takes each agent's top pick and places the order automatically.

Allergy rules aren't left to the model. Code filters out known conflicts before an agent recommends a dish. Uncertain shared dishes are excluded, and uncertain personal dishes are flagged so the diner can check with restaurant staff.

Restaurant owners can search for their restaurant or enter a menu URL. Apify imports the menu, our normalizer Actor tags dishes, and the owner reviews the result before publishing it.

How we built it

iOS app (Swift, SwiftUI). One app has diner and restaurant modes. For tap-to-join, Multipeer Connectivity finds nearby phones and exchanges discovery tokens. Apple's Nearby Interaction then measures the distance between them.
Convex backend (TypeScript). Convex stores profiles, restaurants, menus, tables, orders, and the live agent conversation. It keeps the phones in sync in real time. A Convex workflow runs the negotiation: draft options, vote, settle on an order, and show the conversation.
Two LLM providers behind one wrapper. Bitdeer AI Model Studio drafts dish candidates and writes the agents' speech bubbles. Claude handles voting, onboarding follow-up questions, and taste summaries. We validate replies with Zod and use timeouts and fallbacks so a failed model call doesn't stall the order.
Apify menu pipeline. We use Apify scraper Tasks to bring in menu items, prices, and photos. Our menu normalizer Actor (naoxcv/menu-normalizer | 72HWHrxL8choRGhhA) converts prices, adds diet and cuisine information, and tags allergens. A keyword pass runs first; the model can make an allergen warning stricter, but can't remove it.
Testing. Vitest tests cover safety rules, tap races, onboarding, taste logic, and negotiation. An end-to-end script runs the full backend flow.
Art. Our hand-drawn mascots (shoutout Amy) were vectorized in Python and animated with breathing, blinking, and hopping.

Challenges we ran into

Making a tap feel instant. Discovering a peer, exchanging tokens, and starting UWB ranging felt slow when they happened one after another. We made discovery and token exchange overlap, which made joining feel much smoother.
Race conditions. A QR join and a tap join could collide. Late UWB replies could also undo a newer attempt, and session recovery could bring back the wrong table. We made both join paths compete for one active table on the backend and ignored results from old attempts.
Keeping LLMs out of the safety path. Models can invent menu IDs or miss allergens. We check their replies against actual menu items and use code to decide what an agent is allowed to recommend.
Model latency. Bitdeer tagging timed out on large menus. We made batches smaller, added retries, and moved normalization into an asynchronous publishing flow. Owners can review a scraped draft while it runs; the new menu goes live only when the whole result is ready.

Accomplishments that we're proud of

Tapping phones to join a table actually works with UWB, and it's fun.
The cute design and animations make ordering feel like watching your friends' little characters figure out dinner.
Allergen checks run in code and are covered by tests.
The negotiation finishes with a valid order even if a model call fails.
We built a flow from importing a restaurant menu to watching agents place a group order.

What we learned

Nearby Interaction and Multipeer Connectivity are powerful, but you need real hardware, careful state handling, and a lot of debugging.
The best pattern for LLMs here was: code decides what's allowed; the model helps decide what's good. Validation and fallbacks mattered more than prompt tweaks.
Convex's reactive queries made syncing several phones much simpler. We hardly wrote any sync code.
Using two model providers gave us another path when one call failed.

What's next for TappyTabs

Running pilots at real restaurants with verified menus and connecting to POS systems.
Splitting the bill and showing per-person totals.
Learning from past orders so your agent gets better at representing you.
Giving diners control over what their agent is allowed to say on the shared screen.
Android support.
Removing dish-photo backgrounds to make menus look as playful as the agents.