Inspiration
Service work is the largest labor category on Earth — over 100 million people worldwide. Most juggle multiple jobs, memorize dozens of menus, adapt to constantly changing environments, and are expected to deliver premium, personalized service every single shift. Turnover is sky-high, job stability is low, and yet guest expectations keep rising.
We realized something: no one is building tools for the workers themselves. POS systems serve owners. AI scheduling tools serve management. But nothing exists to make workers faster, smarter, more confident, and more financially secure.
So we built SWANS — the Service Work Assistant & Navigation System. Not to replace workers. But to make workers superhuman.
What it does
SWANS is an AI/AR copilot that follows a service worker across every gig, restaurant, lounge, bar, café, or venue. It:
Converts live table conversations into structured orders using AI
Displays this data on a subtle AR HUD for the worker
Tracks regulars and “SWANS” (VIPs) to elevate guest experience
Helps workers avoid mistakes and reduce cognitive load
Positions floating indicators above guests to guide re-engagement (WIP)
Eliminates the need for pen+pad, POS terminals, or memorization
Empowers workers to deliver premium service anywhere
Helps workers increase tips, speed, and consistency
The result: Faster service, fewer mistakes, higher tips, and premium hospitality delivered by anyone, anywhere.
How we built it
Built a Unity-based AR HUD optimized for subtle, non-intrusive overlays
Used live audio input → AI → structured JSON schema for orders (items, modifiers, quantity, category, VIP tag, special instructions)
Displayed the parsed order in real time on the HUD
Added the first version of S.W.A.N. Indicators — small floating markers meant to attach to unique faces
Created a prototype system for tracking regulars and classifying “SWANS” (our VIP tier)
Attempted automatic table detection to place “engagement markers” in AR space
Experimented with face anchoring to position markers above individuals
Everything was optimized around one principle: augment the human, don’t interrupt them.
Challenges we ran into
Automatically detecting tables in complex lighting and seating layouts
Getting floating engagement markers to anchor stably in physical space
Face detection for positioning SWAN indicators above heads without drift
Creating a clean, minimal UI that enhances service without distracting from it
Ensuring structured order output stayed consistent across varied conversation styles
Avoiding latency when updating the AR HUD with new order data
Designing a worker-focused tool that respects privacy and portability
Accomplishments that we're proud of
Built end-to-end “voice → structured order → HUD display” working inside AR
Designed a consistent, extensible schema for order capture
Implemented the first version of VIP/SWAN tracking
Established a visual language for floating face indicators
Made AR interfaces that preserve human connection by staying subtle and elegant
Proved the viability of a worker-first service copilot
Demonstrated a shift from enterprise tools → personal augmentation tools
What we learned
Workers don’t need automation — they need augmentation
Personal AI adoption is faster than enterprise AI adoption
The best AR UI is nearly invisible and preserves eye contact
Structured outputs + AR overlays = reliable, real-time utility
Building a universal “service OS” requires supporting multi-job workflows
The future of frontline labor belongs to tools that travel with the worker, not the employer
What’s next for SWANS?
Expand the worker profile system to follow workers across venues
Improve marker stability with better face anchoring + depth understanding
Build out a name-recognition & regular-guest memory system
Add micro-gestures for confirmations/corrections
Layer in tip-optimization & service-speed analytics
Push toward a worker subscription model (Superhuman for service workers)
Pilot with multi-job workers across restaurants, lounges, bars, and hotels
Move to lightweight AR form factors and eventually to everyday wearables
SWANS is just the beginning. We’re building the world’s first AI exoskeleton for service workers — a universal copilot that democratizes premium service and helps workers earn more, stress less, and perform at a superhuman level.
Log in or sign up for Devpost to join the conversation.