Specs

Inspiration

Every café, kitchen and shop floor has a binder of standard operating procedures (SOPs), and almost nobody reads it mid-shift. A new barista steaming milk with a shot pulling has no free hand to scroll a PDF, so they guess or stop to ask someone. Asking a chatbot isn't safer: general-purpose AI confidently invents numbers, and a wrong milk temperature or sanitizer ratio is a food-safety problem, not a typo. We wanted the binder to come to the worker, hands-free, without ever making something up.

What it does

Specs puts an employer's own SOPs on Even Realities G2 smart glasses.

For workers: nod twice, ask out loud ("How hot should I steam oat milk?"), and the answer appears on the lens, quoted from the employer's procedure with its section cited, plus a reference picture when one fits. Ambiguous questions get a short clarifying choice. "Walk me through a cappuccino" opens the right section step by step, driven by voice ("next", "back"). If the SOPs don't cover the question, the lens says "Not in your SOPs. Ask your manager to add it.", so every gap becomes a request for a new procedure.

For employers: a web portal to upload SOPs as PDFs, Word files or photos of paper pages. AI converts them into step-by-step drafts, every step is checked against the source, and a supervisor reviews and approves before workers see it. Supervisors pair glasses, assign tasks and watch a live board of who is on which step.

How we built it

Glasses app: TypeScript on the Even Hub SDK. It runs in the Even app on the phone, using the G2's display, microphone and motion sensor. Double nods are detected from the G2's accelerometer, tuned on real glasses. Server: Python and FastAPI in Docker, with Caddy for HTTPS, on a Vultr VM. AI: Google Gemini for speech-to-text, 768-dimension embeddings, structured cited answers, and multimodal conversion of uploaded documents. Database: Tiger Cloud Postgres. pgvector and full-text search are merged with reciprocal-rank fusion for hybrid retrieval, and TimescaleDB hypertables store per-request latency and live progress. Trust layer: a deterministic verifier with no AI in it. An answer reaches the lens only if every quote exists in the cited SOP and every number and unit appears in a quote. Off-topic questions are refused by a similarity gate before Gemini is even called. Observability: Sentry traces one question end to end, from the nod to the answer on the lens.

Challenges we ran into

Hardware surprises: the G2's motion sensor is a gravity accelerometer, not a gyroscope. Nods worked, but our planned head-shake gesture was physically undetectable, so we redesigned around double nods and taps. Raw greyscale images rendered in the simulator but not on real glasses, so we switched to pre-baked PNGs. Keeping AI honest without making it useless: a verifier that's too strict refuses good answers, and one that's too loose lets wrong numbers through. We tuned it so a changed digit or a converted unit always fails, while spacing and punctuation differences don't. A 576×288 green display: fitting answers, warnings and pictures on a tiny monochrome lens without ever cutting off a safety warning or a value. API rate limits: free-tier Gemini quotas ran out mid-testing, so we built paced evaluations and fallbacks for quota errors.

Accomplishments that we're proud of

It runs on real G2 glasses: nod, ask, and a verified answer on the lens. 0 unverified answers in our evaluation on the live system, and 0 invented values when converting uploaded SOPs into steps. A median of about 1.2 seconds of server time per answer, across live requests. A full employer loop, from upload to verify, approve, assign, and track, working end to end.

What we learned

• Generative AI is most trustworthy when it isn't the final decision-maker. Letting Gemini draft and letting plain code decide gave us both fluency and safety.
• Hybrid search beats either method alone. Workers ask by meaning ("how hot is the milk") and by reference ("SOP-210 4.2").
• Hardware has to be tested early. The simulator and the real glasses disagreed on gestures and images, and only real-world testing caught it. ## What's next for Specs
• Field trials with baristas, measuring nod accuracy and false triggers over full shifts.
• Per-supervisor accounts and SSO for the employer portal, along with stronger workspace security.
• Supervisor alerts when a worker logs a deviation or keeps hitting "not in your SOPs", so missing procedures get written fast.
• More industries and languages: manufacturing, healthcare support and maintenance, where hands-free, cited instructions matter most.

Built With

• docker
• even-hub
• even-realities-g2
• evenhub-simulator
• fastapi
• gemini-api
• gemini-embedding
• hybrid-search
• imu
• pgvector
• postgresql
• psycopg
• pypdfium2
• pytest
• python
• rag
• react
• sentry
• tiger-cloud
• timescaledb
• typescript
• ubuntu
• vector-search
• vite
• vultr