Umori

Umori turns any camera into a live mood dashboard for cafés, restaurants and stores — anonymous, aggregate sentiment by hour and zone. No faces stored, no identities. Know how your room feels

Comment

Umori — know how your room feels

Inspiration

Over the past year and a half I've been building a computer vision platform for real-world operations — people detection, tracking, presence analytics running on ordinary cameras in ordinary buildings. Along the way a gap started bothering me: every venue measures how many people walk in, and nobody measures how it felt to be there. Foot traffic is a solved problem. Experience is invisible.

At the same time, the obvious way to make it visible — pointing emotion recognition at customers — is exactly the kind of thing that goes wrong fast. Identity tracking, stored faces, per-person profiles: creepy by default.

Umori is my attempt to prove the interesting version is buildable: a system that understands the mood of a space while being structurally incapable of knowing anything about a person. Aggregate emotion, zero identity. If the privacy constraint is baked into the architecture rather than the privacy policy, the whole product category becomes something a café owner can actually feel good about running.

What it does

Umori turns any standard camera into a live mood dashboard for physical spaces — cafés, restaurants, retail floors.

  • Detects faces in the video stream and classifies momentary expressions (happy, neutral, tense, tired...) — then immediately discards the frame and the face. No images stored, no embeddings, no identities, no per-person tracking.
  • Aggregates those momentary signals into a rolling sentiment index — overall, by zone, and over time.
  • Streams it to a live dashboard: how the room feels right now, how the mood curve moved through the day, which zone dipped at 6 p.m. when the queue built up.
  • Only aggregates ever leave the edge. What crosses the wire is numbers like "zone B, 18:00–18:05, n=14, sentiment 0.62" — nothing that could reconstruct a person.

How we built it

Camera frames go into a Python edge service — OpenCV capture, YuNet face detection, an ONNX FERPlus expression classifier — which discards every frame and face crop the moment it's processed. Only an aggregate dict per zone and time window ever leaves that layer. A FastAPI service turns those aggregates into rolling 5-second live windows and 1-minute SQLite rollups, served over Server-Sent Events (/api/live) and REST (/api/history, /api/zones). A Next.js dashboard consumes that stream directly: a live sentiment gauge, per-zone mood badges, and a toggleable history timeline, styled to a dark/cream/sand/orange brand system. A DEMO=1 replay mode runs the same API and dashboard off a bundled recorded aggregate file, so anyone can test the whole product with zero hardware. Deployed in production as three services: marketing site (Cloudflare Workers), API (VPS, Docker/Nginx/Certbot), and dashboard (Node.js hosting).

How Codex fit in:

  • Block 1 — pipeline spine: Human guidance set the clean-room architecture, privacy boundary, and GPU safeguards. Codex audited dependencies, selected and pinned YuNet and FERPlus, implemented capture, anonymous detection and classification, rolling aggregation, FastAPI, SQLite, SSE, privacy tests, and milestone commits.
  • Block 2 — dashboard: Human guidance supplied the brand system, required views, and visual review. Codex scaffolded the Next.js dashboard and built the live gauge, zone badges, timeline filters, SSE/history integration, CORS protections, and metadata.
  • Block 3 — demo/replay: Human guidance required DEMO=1 to run with no camera or Ollama, through the real ingestion boundary. Codex built the deterministic replay engine, routed it through the same aggregation/persistence/SSE path, and verified the judge flow end to end.
  • Polish and shipping: Human guidance made the deployment and cut-line calls. Codex handled the API-only Docker/Nginx configuration, deployment docs, and final live-recording checks.

Where GPT-5.6 fit in: GPT-5.6 powered Codex throughout — turning the build spec into milestones, writing and testing the privacy-safe edge pipeline, implementing the FastAPI/SQLite/SSE/CORS/demo-replay layers, building the dashboard, and authoring the regression tests that enforce the privacy boundary. GPT-5.6 built the system; it isn't part of it. Runtime inference runs entirely on pinned YuNet and FERPlus ONNX models at the edge — no calls to GPT-5.6, OpenAI APIs, or Ollama at runtime.

Challenges we ran into

Keeping the privacy boundary structural rather than just promised meant testing for absence — asserting the SQLite schema has no BLOB columns, asserting the API payload schema rejects image/byte/embedding fields entirely, instead of just checking the feature worked. We also shared a single 6GB GPU machine, so both ONNX models were forced to CPU inference, with a preflight check against Ollama before every camera or model run so nothing collided. Per-frame expression classification is noisy by nature — solved by sampling at 2–4fps and letting the rolling-window aggregation smooth it, rather than chasing per-frame accuracy.

Accomplishments that we're proud of

  • A working camera-to-dashboard loop built in 48 hours.
  • A privacy model enforced by architecture, not by promise: the system cannot leak what it never stores.
  • A full production deployment — three live services (marketing site, API, dashboard) across three different hosts, wired together with correct CORS and DNS, not just a local demo.
  • Real-time face detection running against a live webcam, not just simulated or replay data, proving the vision pipeline actually works outside DEMO mode.

What we learned

Per-frame emotion classification is noisy and always will be — and that turns out not to matter. Individual readings are unreliable; the aggregate is surprisingly stable and genuinely informative. Umori never claims "this person is unhappy." It claims "this corner of the room trended down between 6 and 7 p.m." — a much weaker claim per person, and a much more useful one per venue.

Working agentically with Codex meant defining the verification test before the implementation — that discipline is what let an entire block's tests and a live camera run pass without us hand-debugging model internals.

What's next for Umori

  • Longitudinal insights: mood vs. staffing, playlist, layout changes — did the intervention work?
  • Multi-camera zone stitching for larger venues.
  • An operator alert layer ("zone B trending down for 20 min") instead of a dashboard you have to watch.
  • Pilot deployments in real venues to validate the aggregate signal against what owners and staff perceive.

Anonymous. Aggregate. Ethical.

Built With

Share this project:

Updates