Inspiration
Manual retail auditing is broken. Staff are forced to walk aisles with clipboards, making the process slow, tedious, and error-prone. By the time a traditional audit is completed, stockouts have already drained sales, contributing to the 4–5% of retail revenue lost annually due to out-of-stocks (OOS). We realized that no current system actively knows a shelf is empty until a customer complains or staff manually stumble upon it, leaving misplaced items completely invisible. We wanted to bridge this real-time gap by turning standard security and aisle cameras into intelligent, automated auditors.
What it does
Shelf Sight is an end-to-end computer vision and real-time shelf auditing system. It captures RTSP video frames from standard camera feeds, uses advanced detection models to find products and gaps, and matches them against a Digital Planogram JSON. If an empty slot, low stock, or a misplaced item is detected, the system immediately triggers automated alerts via MQTT or Telegram to notify store staff for instant re-stocking and displacement correction.
How we built it
We built this project in just one day using a robust multi-layer tech stack: Computer Vision: We utilized Yolov8 and OpenCV for real-time product and gap detection, backed by DINOv2 and FAISS. Backend: Built using Python and JSON, incorporating temporal logic to stabilize detections and prevent false alarms. Frontend: Developed a real-time dashboard using Streamlit to showcase OOS, low stock, and misplaced item metrics instantly. Deployment: Containerized with Docker for edge deployment on hardware like NVIDIA Jetson.
Challenges we ran into
Real-time Performance vs. Accuracy: Balancing inference speed with detection accuracy on standard feeds required careful optimization of our computer vision pipeline. Planogram Mapping: Mapping 2D bounding boxes from camera frames to a structured 3D/grid-based Digital Planogram JSON required precise spatial logic. False Alarms: Handling momentary occlusions (like a customer's hand reaching for a product) required implementing temporal logic to avoid spamming staff with false restock alerts.
Accomplishments that we're proud of
- Built a fully functional end-to-end MVP in just 1 day.
- Successfully mapped a live webcam feed to a 3-shelf x 4-slot digital planogram.
- Integrated instant live Telegram notifications so staff get alerted the moment a stockout occurs. ## What we learned
- The importance of robust edge computing and lightweight models (like YOLOv8) for industrial retail environments where latency matters.
- How critical spatial mapping and digital planograms are for translating raw computer vision bounding boxes into actionable business intelligence. ## What's next for VisionShelf:SHELF SIGHT,STOCK RIGHT Phase 2: Fine-tuning models to scale support for over 10,000+ unique SKUs. Phase 3: Integrating the software with autonomous roaming robots for continuous store-wide coverage. Phase 4: Deep API integrations with enterprise retail systems like POS and SAP.
Log in or sign up for Devpost to join the conversation.