Desk Sentinel

• 

  Detecting an object.

What it does

Desk Sentinel is an app for people who want a clean desk without constantly checking it themselves. A fixed webcam watches the desk, compares it with a saved clean reference, and identifies objects that were added or moved.

The app uses two vision layers. TensorFlow.js and spatial image differences run locally for responsive tracking. Every ten seconds, the Codex app-server uses GPT-5.6 to compare the saved reference with the latest camera image, even when a person is moving in frame. GPT-5.6 identifies the current object, checks whether it is resting on the desk, and returns a separate bounding box for each actionable item.

When an object remains on the desk, Desk Sentinel opens a full-screen Windows interruption. Instead of showing a vague “clean your desk” message, it displays the object name and a focused image of exactly what must be removed. The interruption remains until the user removes the object and presses I cleaned it. That button checks two fresh camera observations before releasing the screen. You can't lie!!!

How I built it

Local tracking runs in the renderer with TensorFlow.js, COCO-SSD, and spatial image differences. A placement tracker tolerates detector jitter and requires continuous desk contact before escalating. The Node.js server validates strict judge output, reconciles allowed objects, suppresses duplicate alerts, and controls the Electron interruption window.

GPT-5.6 is used through the Codex app-server as the semantic judge. It receives the saved reference, the current frame, and optional changed-region crops. Deterministic code remains responsible for timing, privacy, persistence, cleanup confirmation, and notification policy.

Challenges

Real rooms are much harder than still-image demos. Detector boxes move slightly between frames, hands cross the desk, the camera view is mirrored, and a new object can replace something that used to occupy the same location. We added jitter-tolerant tracking, current-image identity checks, moved-object handling, and recurring verification that continues while a person moves. It should be very good in factory to check unnecessary objects.

How Codex helped

Codex accelerated the complete build-and-review loop: application architecture, local tracking, Codex app-server integration, strict schemas, Electron interruption behavior, privacy controls, tests, visual readback, packaging, and real Windows verification.

The final repository includes 56 automated tests, synthetic new/replaced/moved-object evidence, desktop and mobile visual checks, privacy validation, an installer, and a portable Windows build.

Please Try it

Download the installer or portable build from the GitHub Release. Save a clean desk reference, enable the Codex verification backend, start the guard, and place an object on the desk. Remove the detected item and press I cleaned it to release the interruption.

Built With

• coco-ssd
• codex
• codex-app-server
• electron
• gpt-5.6
• javascript
• node.js
• tensorflow.js