Inspiration
Clothing sizing is inconsistent across brands and regions, making online shopping frustrating and leading to high return rates. We wanted to build a tool that could automatically measure garments from a simple photo, giving shoppers and retailers an objective, data-driven way to understand clothing dimensions before purchasing.
What it does
MEASURE is a real-time cloth segmentation and measurement system. A 1080p webcam streams a live feed to a web interface accessible over a local network. The user clicks a button to capture a photo, which is passed through a custom-trained deep learning model that segments clothing into categories (top, bottom, shoes, etc.) and calculates the length and width of each garment in centimetres — calibrated using the person's height. The result is displayed side by side with the original photo, with a history strip of previous captures.
How we built it
Hardware: Rubik Pi 3 (Qualcomm QCS6490, 12 TOPS NPU) running Ubuntu 24.04, with a Logitech Brio 105 USB webcam Camera streaming: mjpg-streamer serving a live MJPEG feed over HTTP on the local network AI model: Custom PyTorch segmentation model (DeepLabV3-ResNet50 backbone) trained to segment 7 clothing classes, with a measurement pipeline that calculates garment dimensions by scaling pixel spans against a known person height Backend: Flask web server exposing a /capture endpoint that grabs a snapshot from mjpg-streamer, runs inference, and returns the annotated image and measurements as JSON Frontend: Vanilla HTML/CSS/JS with a dark industrial UI, live camera feed embed, measurement cards, and capture history Connectivity: SSH over WiFi for development, with the board accessed via serial console (Micro USB / CH343 UART) during initial setup
Challenges we ran into
GPIO debouncing: Getting a physical push button to reliably trigger captures was tricky — the internal pull-up on the Rubik Pi 3's GPIO was too weak to recover cleanly, causing the line to latch LOW after the first press Camera device conflicts: The Brio 105 registers on /dev/video1 rather than the expected /dev/video0, and mjpg-streamer and fswebcam couldn't share the device simultaneously — solved by using mjpg-streamer's built-in snapshot endpoint instead of fswebcam GPIO line mapping: The 40-pin header's physical pin numbers don't map directly to GPIO line numbers in the kernel — required using gpiodetect and gpioinfo to find that physical pin 15 maps to line 24 on gpiochip4 Checkpoint loading: The inference package structure shadowed the top-level inference.py file, causing import errors that required renaming the file to resolve Network setup: Initial WiFi connection failed with a security detection error, requiring explicit WPA-PSK flags in nmcli
Accomplishments that we're proud of
Getting a full end-to-end pipeline running on edge hardware — from live camera feed to AI inference to web interface — entirely on a single-board computer Training a custom cloth segmentation model that correctly identifies and measures garment classes Building a clean, production-quality web UI that works on any device on the local network with no installation required Successfully debugging low-level Linux GPIO, V4L2, and USB issues from scratch on an unfamiliar board
What we learned
How to configure and interface with GPIO on the Rubik Pi 3, including the difference between physical pin numbers and kernel GPIO line numbers How V4L2 device enumeration works and why USB cameras don't always appear on predictable device nodes How to serve a live camera stream over HTTP using mjpg-streamer and consume it from a web frontend How to wrap a PyTorch inference pipeline in a Flask API and connect it to a hardware camera system How to access and develop on embedded Linux hardware using serial console (UART) and SSH when no display is available
What's next for Chipmunks
Physical button integration: Resolve the GPIO debouncing issue with a hardware capacitor and re-enable physical button-triggered captures On-device NPU acceleration: The Rubik Pi 3's Qualcomm Hexagon 770 NPU supports 12 TOPS — migrating the model to run via Qualcomm AI Hub would dramatically speed up inference Reference object calibration: Instead of requiring a manual height input, detect a reference object (coin, credit card, ruler) in frame to automatically calibrate measurements Multi-garment database: Store measurements over time and export as CSV for use in size recommendation systems Public deployment: Expose the interface via Cloudflare Tunnel for access outside the local network
Log in or sign up for Devpost to join the conversation.