Verity - Your Personal Restaurant Helper

• 

• 

• 

• 

• 

• 

• 

Inspiration

Anyone who has worked a fine-dining floor knows the moment: a plate of hot halibut sits at the pass while the server who should run it is across the room, getting waved down by another table. The expo is shouting, the manager is guessing where everyone is, and the whole night is held together by memory, eye contact and luck.

Restaurants have plenty of software, but almost all of it lives on a screen someone has to walk to. Servers can't stare at a phone in front of guests, and their hands are full anyway. When we got our hands on the Even Realities G2, a pair of glasses with a small, quiet heads-up display, we saw the right shape for the problem: put the one thing a server needs next directly in their line of sight, and nothing else.

Floor Sense (with its voice assistant, Verity) is what came out of that idea: the restaurant's task queue on smart glasses, built to stay out of the way until it's needed. It ships as an Even Hub app named Floor Sense.

What it does

Floor Sense connects three groups who usually only talk by shouting: servers, managers and the kitchen.

For servers: one task at a time, in their line of sight. A server sees a single card like RUN · T12 · 2x halibut, seats 1+3 with +2 queued in the corner. When a server has nothing to do, the glasses go fully blank so they don't distract, and they wake on a tap, a new task, a queue item, an alert or a finished clip. Voice commands and clipping keep working while the screen is blank. A tap accepts, a second tap marks it done. A long hold hands the task to someone else, and before accepting they can decline it. If nobody accepts a task within 30 seconds, it escalates to the next server and then to the manager. If an accepted task is still open after 5 minutes, the manager is warned and the task is rerouted.

Hands-free: "Verity" voice commands for every gesture. Servers' hands are full, so every gesture also has a spoken command behind the wake word "verity": verity accept, verity done, verity handoff Ben, verity revert, verity clip, verity table 4 and more. Because every command has to start with the wake word, things guests say never trigger anything. Anything after "verity" that isn't a known command goes to Gemini, which works out what the server meant. "verity give it to Ben" becomes a handoff to Ben, and if Gemini can't work it out, the glasses say Didn't get: … rather than guessing.

Order taking without a notepad: clip the last two minutes. The server takes the order the way they always have, talking to the table. The glasses keep a rolling 2-minute audio buffer in memory only. Nothing is saved, and nothing leaves the glasses unless the server clips it. A swipe back (or verity clip) sends that clip to ElevenLabs Scribe, and an agent turns it into:

• the order by seat, with alterations, allergies and notes, plus a ? line for anything it couldn't make out,
• a short summary of what the table said,
• suggested answers to diner questions, drawn only from the restaurant's menu data (? when the menu can't answer).

A small "Verity: processing" animation shows while it works. Nothing reaches the kitchen until the server taps to confirm, so a person stays in the loop.

For managers: exceptions, not noise. A manager wearing the glasses sees only problems: an unacknowledged task, a slow-task reroute, T4 waiting 14 min on mains, VIP seated, and 86 alerts when a dish runs out, which go to every server at once and block the dish from new orders. A swipe brings up a floor minimap of up to 8 tables, each with a status glyph and a timer.

On the laptop: a live manager dashboard and a kitchen page. A live dashboard, designed in Paper, has a dispatch bar, a floor map, an alerts rail with one-click reroute, a task board with countdown rings, and side panels that slide open to show the staff list, a log of which server the router picked and why, and, for any table, its latest order summary and the diner questions still unanswered. A kitchen page receives the confirmed tickets, and "Food up" calls a runner.

Table requests and statuses by voice. A server can say verity table two wants the check (or clip it), and a Request line appears on the glasses. One tap turns it into a task, which the router assigns and the dashboard shows. Statuses work the same way: verity table 2 paid, table five is free or T4 seated four set the table's status after a confirm tap, and the dashboard and the manager's minimap update live with who set it. A status that means work, like wanting the check, also creates a task.

For new servers: ramp mode. Saying verity ramp shows each dish on the card with its components, allergens and a wine pairing (for example, Pan-Roasted Halibut with Sancerre from the Loire, and a table's allergy shown in capitals next to its seats), and verity ramp on adds a one-line dish hint to every run card, so a server in their first weeks can learn the menu while they work.

Smart dispatch. Gemini (Flash-Lite, with a hard 3-second cap) routes each new task to a server, using each server's real accept-to-done times. Fixed rules take over on any failure (no key, a timeout, a used-up quota or a bad answer), and they also handle escalation and bounces. Food runs from the pass follow a heat-aware priority engine (see How we built it).

How Devin (Cognition) powered Floor Sense

Devin wasn't a side tool for us. It was the engineer who built most of the product, with Claude acting as its tech lead.

Claude drove Devin through the Devin MCP. We split the project into two lanes: the restaurant brain (backend, routing, web pages) and everything the wearer sees and says (glasses app, HUDs, voice). Devin built nearly all of the glasses-and-voice lane. Claude, connected to Devin's official MCP server, read our build plan, wrote each Devin session's brief (devin_session_create), followed up mid-task (devin_session_interact), and reviewed what came back. Devin cloned the repo, wrote the code and tests, and opened a pull request for each piece.

Devin wrote most of the codebase. When the build was done, the repo had 347 commits and +44,349 / −3,928 lines (net +40,421). Of the 285 non-merge commits, Devin authored or co-authored 247, and its commits go out under its own Devin AI identity so anyone can audit them. Devin's PRs include:

• the glasses app's gesture and voice controls, the controls menu, handoff and revert,
• the clip pipeline: the order card, the summary, menu-grounded answers and verity table N,
• the switch to a 2-minute clip with the processing animation,
• voice in a crowd: speech-recognition keyterms, sound-alike matching, a loudness gate, speaker labels, and an automated crowd-noise benchmark to prove it (numbers below),
• manager exceptions and 86 alerts, Sentry in the glasses app, ramp mode for new servers, the neighbouring-table leak filter, and the dinner-rush simulator study.

Devin closed the loop from real-world testing to fixes. When we tested a build on the real G2 and found problems (swipe directions that felt backwards, a menu with no way out, a cluttered queue), Claude wrote them up as a Devin task and Devin shipped the fix as a PR, often while we were still testing other parts.

Devin also runs our merges. Claude runs a merge checklist on every PR: the head commit, the base branch, a scan of added lines for secrets, a check for stray files, and the tests. Devin then merges the queue in order and fixes any conflicts, rerunning the tests before each merge.

In short, two humans set direction, Claude turned it into precise engineering tasks and reviewed the results, and Devin did the long-horizon building. That is how a hackathon team ended up with a glasses app, a voice pipeline, a routing backend and a live dashboard, all tested and all running together.

How we built it

Built at LA Hacks. Everything in Floor Sense was built during the hackathon: the glasses app, the backend, the voice pipeline, the router, the simulator and the web pages, starting from an empty repo. The only thing we brought in was an AI coding skill we had written beforehand with notes on building apps for the G2 glasses.

The glasses app (TypeScript, Vite, @evenrealities/even_hub_sdk). Even Hub apps run as a WebView inside the Even phone app, which draws to the glasses' 576×288 monochrome green display. The glasses app holds no business logic: it renders what the backend sends and reports gestures and audio. Cards are updated in place to avoid flicker, and the app re-syncs from the backend whenever the phone app brings it back to the foreground. The glasses and backend talk through a frozen message contract (glasses/src/contract.ts), which let both lanes build in parallel against a stand-in backend from the first hour.

One task, end to end:

The backend (Python, FastAPI, WebSockets). One task state machine (queued → sent → accepted → done) is the single source of truth for glasses, phones, the dashboard and the kitchen, so any user can drop and reconnect without losing the queue. Escalation, bounces, handoff, one level of revert and the 5-minute reroute timer all live here. Every task event is written to Postgres on Tiger Data as a time series, and GET /stats gives each server's average time from accept to done by task kind, which Gemini weighs when it routes a task.

Heat-aware run priority. Every plate waiting at the pass gets a priority, and the highest-priority run goes to the next free server. Course comes first, so any appetizer outranks any main. Within a course, temperature and time at the pass decide:

$$ \text{priority} = \big(\,\text{course},\;\; \text{temperature} + \text{decay} \times \text{minutes waiting}\,\big) $$

The two parts are compared in order, so course always wins. Course is 3 for appetizers, 2 for mains and 1 for desserts. Temperature is 3 for hot, 2 for warm, 1 for room temperature and 0 for cold. Decay is 3 per minute for hot plates, 2 for warm and 1 for the rest, so hot plates climb fastest because they degrade fastest. Any plate waiting more than 5 minutes jumps to the top, so cold plates never get stale. Pairing rules sit on top: a table's course leaves together, hot plates for the same section ride together (up to 3 per server), and a cold plate never delays a hot one. A dinner-rush simulator replays a full service under first-come-first-served (FIFO) and under the priority engine, then prints the two side by side. A sweep runs 20 simulated nights.

From conversation to kitchen ticket:

Voice (ElevenLabs). The glasses mic records 16 kHz PCM, and the glasses app keeps the last 120 seconds in a ring buffer. A clip goes to Scribe for batch transcription with speaker labels, and an ElevenLabs agent in text-only mode calls our backend's order tools (add_item, remove_item, modify_item, set_allergy, confirm_order, cancel_order). Dishes must match data/menu.json, and alterations are checked against what each dish allows. Live verity commands are matched against a fixed word list first, with no LLM in the loop, so they're fast; only unknown phrases go to Gemini.

The web (React 18, Vite, GSAP, Lenis, React Bits). We designed the manager dashboard and a five-scene scroll story in Paper, then built them from the Paper file. The dashboard shares the same client code and live socket as the original organizer and kitchen pages.

Observability (Sentry). We use three Sentry products beyond error monitoring. Tracing follows a task across the system: backend spans cover dispatch and delivery, and the web pages trace their requests in the browser. Session Replay records the organizer and kitchen pages, so we can watch what a manager did right before something went wrong. Logs come from both the backend and the glasses app.

How observability shaped the build. One trace followed a task from the Dispatch click through dispatch, delivery, accept and done, and it showed the trace stopping at the glasses. So Devin added spans for gestures, clips and incoming tasks to the glasses app, and one trace now runs from the dashboard to the wearer and back. Tracing also told us what our tooling costs: we timed 200 tasks with and without Sentry and Tiger Data. Sentry adds about 0.2 to 0.4 ms per action and Tiger Data adds nothing we could measure, so we kept both on.

Hosting. The backend, with Tiger Data and Sentry, and the manager dashboard run on a Vultr server, so the glasses build works from any network. For the real-G2 builds we packaged .ehpk files, uploaded them to Even Hub as private Beta builds and installed them through the Even app. During development, a Tailscale Funnel gave the glasses a fixed wss:// address for a laptop backend. To try it yourself, the repo's README covers running it locally, installing it on a G2 through the Even Hub Beta channel, every gesture and voice command, and the tests.

Challenges we ran into

Voice in a crowded room. We're presenting on a loud floor, and a noise test with 4 talkers in the background showed the weak spot: once other people's talk got anywhere near the wearer's volume, almost no commands landed. Every miss came from other people's words getting into the transcript, not from "verity" being misheard. Keyterms, sound-alike matching, a loudness gate and speaker labels fixed most of it. We measured the result instead of guessing, with 28 "verity" commands at each noise level:

• 12 dB quieter than the wearer: 5/28 before, 28/28 after
• 6 dB quieter: 1/28 before, 27/28 after
• 3 dB quieter: 0/28 before, 26/28 after
• equally loud: 0/28 before, 9/28 after
• 3 dB louder: 0/28 before, 1/28 after

False triggers stayed at 0, and a command shows up about 1.05 s after the server finishes saying it. Order clips had a related problem: a neighbouring table's dishes sometimes leaked into the order. Across 40 test clips, a filter Devin added cut those leaks from 8 to 4, at the cost of dropping one of the table's own dishes out of 120. That is better but not solved yet. Someone right next to you talking as loudly as you still beats the system too, so on stage we lead with tap and swipe.

Our simulator proved our own hypothesis wrong. We expected priority dispatch to cut the time hot plates wait at the pass. Across 20 simulated nights it didn't: the median wait was 28.4 s with priority versus 11.3 s with FIFO, and the 90th percentile was 119 s versus 56.7 s. The cause is our own fine-dining rule. A course leaves only when its last plate is up, so every hot plate waits for its slowest neighbour no matter which planner runs, and that wait alone already matches FIFO's median. With only two servers and a 3-plate limit per runner, courses of 4 or more plates also have to wait for two free servers. Measuring the right thing showed where priority does win. From the moment a course is complete, the hot course is whole at the table in a median of 87 s with priority versus 104 s with FIFO, and priority wins on 19 of 20 nights. We wrote this up instead of tuning the numbers until they looked good.

An always-on mic at a dinner table. Recording guests is a real privacy problem. Our answer is a buffer that lives only in the glasses' memory, is never saved, and leaves the glasses only when the server deliberately clips it, plus a wake word so ordinary conversation can't trigger anything.

Designing for 576×288 pixels of green. The G2 has no font-size control and very little space. Every screen had to be readable at a glance in the middle of service, which pushed us to show one card at a time, an exceptions-only manager view and a minimap instead of a full floor plan.

Real hardware vs. the simulator. Gestures felt different on the real temple touchpad (we relabelled swipes as "fwd" toward the eyes and "back" toward the ear), and the phone app reloads the WebView when it goes to the background, so the app has to rebuild its state from the backend every time it comes back.

Networking at a venue. The venue Wi-Fi isolates devices, so the glasses' phone couldn't reach a laptop. We routed everything through a public backend over cellular, and because each build has to whitelist its backend host when it's packaged, we needed a stable address before shipping a build.

Keeping two lanes, two humans, Claude and Devin in sync. A frozen message contract, a shared build plan and a strict merge checklist kept parallel work from colliding.

Accomplishments that we're proud of

• It runs on real glasses. We tested builds on a real G2 through Even Hub's Beta channel, and every tap and swipe reached the backend within a second.
• Fast where it matters: about 47 ms from a task being dispatched to the card appearing, and Gemini routes a task in 0.5 to 0.8 s, with rules as a safety net.
• Voice that holds up in a crowd, with 27 of 28 commands heard against background talk 6 dB quieter than the wearer (up from 1), and zero false triggers.
• Whole courses reach the table faster: in the rush simulator, priority dispatch gets a finished hot course to the table in a median of 87 s versus 104 s for FIFO, and it wins on 19 of 20 simulated nights.
• A human stays in the loop: nothing an AI drafts, whether an order, a cancellation or a dispatch, reaches the kitchen without a tap.
• An AI-built codebase you can audit: Devin authored or co-authored 247 of 285 non-merge commits, every change landed as a reviewed PR, and 565 automated tests gate every merge, all passing on the final build (138 backend, 154 voice, 17 simulator, 36 web, 19 dashboard and 201 glasses).

What we learned

• Measure, don't guess. The crowd-noise benchmark changed what we built: we stopped tuning recognition and started keeping other people's voices out of the transcript.
• Pick the metric your guests feel. Plate wait at the pass made priority dispatch look worse. Time until the whole course is at the table, which is what diners actually notice, showed that it helps.
• On a heads-up display, less is the feature. The best screen is usually a blank one, and the hardest design calls were about what to leave out.
• Agents work best with a clear spec and a gate. Devin was most effective with a precise brief, a frozen contract to build against, and a merge checklist at the end. Claude writing the brief and checking the result, with Devin doing the build, turned out to be a very productive pairing.
• LLMs belong at the edges of real-time systems. Gemini routes tasks and ElevenLabs drafts an order, but every time-critical path has a deterministic fallback with a hard time cap.

What's next for Floor Sense

• Smarter course holds and runner pairing, so the plates waiting on the rest of their course stop costing time at the pass.
• A real pilot in a restaurant during a full dinner service, to tune the priority weights on real tickets instead of simulated nights.

Built With

• claude
• css
• devin
• python
• typescript
• vultr