Ubik | Devpost

Every supermarket manager knows an uncomfortable truth: the central system and the physical shelf rarely align. While digital inventory records or planograms suggest everything is in order, the reality in the aisles is different. Price tags show outdated figures, gaps appear where products should be, and items end up jumbled together.

We were inspired to create Ubik after discovering that these errors remain completely invisible to the system until a sale is lost or a customer complains—a critical issue in Mexico, given the fines imposed by PROFECO. Manual auditing is slow, relies on sampling, and lacks a vital element: it fails to quantify the cost of the error. We wanted to transform a tedious visual task into a financial and operational tool that brings order to the chaos by focusing on the financial stakes involved.

How We Built It We developed a four-stage processing workflow (Capture, Read, Cross-reference, Act) by integrating multiple artificial intelligence models and business logic:

Frame Selection and Vision: We used OpenCV to extract the sharpest frames based on Laplacian variance.

Reading and Detection (OCR & AI): We implemented PaddleOCR (PP-OCRv5) to read prices and SKU codes, applying a voting system across multiple frames to ensure maximum accuracy. To detect products, gaps, and facings, we trained a YOLO11n model using synthetic scenes (shelves) and the SKU-110K dataset.

Visual Validation: We integrated DINOv2 to assess the similarity between the physical product image and the digital catalog, and to detect items placed incorrectly.

Rules and Calculation Engine: Processed in Python, the system compares physical detections against the planogram and inventory/pricing CSV files.

Interface and Assistance: The dashboard was built using Next.js, TypeScript, and Tailwind CSS, operating entirely locally (without cloud dependency) to maximize speed. Additionally, we integrated Gemini with RAG to create an assistant that provides natural language responses based on the run's data.

Achievements We Are Proud Of Extreme Precision: In our 4K demo featuring 8 planted errors, we achieved 100% accuracy (11/11) in label reading and pricing, as well as 100% accuracy in gap detection with zero false positives. Our proprietary shelf detector achieved an mAP50 of 0.995.

Local Processing Speed: We processed a 10-second clip in just 19 seconds using only a laptop CPU (17.6 seconds with GPU).

Live "Jury" Mode: We developed a feature where a panel selects an error at random, and the system processes the live video feed in front of them—identifying the issue and highlighting it on the digital twin in just 9 to 13 seconds.

Real-World Validation: We manually set up a physical shelf without using markers and recorded it with a smartphone; Ubik detected 7 of the 8 planted errors, instantly calculating a daily value at stake of over $3,500 MXN.