Inspiration
Walk into any electronics assembly line, automotive parts plant, or small-scale manufacturing unit in India, and you will find the same bottleneck: a worker standing at a bin of mixed components, manually picking and sorting them into the right trays. It is repetitive, exhausting, and error-prone. Misclassify one part, and the entire downstream assembly rejects. Scale up production, and you need more people doing the same mind-numbing task.
We looked at the numbers: manual sorting typically runs at 85–92% accuracy, with fatigue-driven errors climbing sharply after the third hour of a shift. For a line sorting 1,000 parts per shift, even a 5% error rate means 50 misclassified parts and hours of rework. Meanwhile, industrial robot arms have become dramatically cheaper and more accessible—open-source platforms like LeRobot have put a capable cobot within reach of any lab or small factory.
We asked ourselves: why is flexible, vision-guided sorting still locked behind months-long integration projects? That question became SortMaster. We wanted to prove that a vision-guided cobot that sees, picks, and sorts can be prototyped in 24 hours using open-source tools—and scaled to real factory floors with just a small dataset.
What it does
SortMaster is a vision-guided collaborative robot (cobot) that autonomously picks and sorts mixed parts into correct bins based on type, color, or defect status.
The workflow is simple and fully autonomous:
See — An overhead camera captures the workspace. A fine-tuned YOLO model detects and classifies each part, drawing bounding boxes and confidence scores in real time.
Decide — The system maps each detected part to its correct destination bin and calculates the pick coordinates using a kinematics solver.
Pick & Sort — The robotic arm moves to the target, closes its gripper, lifts the part, and places it into the correct bin.
Report — A live dashboard shows the camera feed with detections, per-bin counts, and a throughput metric (parts sorted per minute, classification accuracy).
How we built it
We deliberately separated perception, decision-making, and hardware control so each layer could be built and tested independently—critical for surviving a 24-hour build.
Hardware:
A low-cost LeRobot SO-100/SO-101 follower arm with a parallel jaw gripper
An overhead USB webcam for a stable top-down view of the workspace
Three labeled sorting bins and a set of distinct colored parts for the demo
Perception (Computer Vision):
YOLOv8 fine-tuned on a small curated dataset (~150 images) of our demo parts
OpenCV for camera capture, frame preprocessing, and bounding-box overlay
A color-detection fallback using HSV thresholding in case the model underperformed on the day
Decision & Control:
A mapping layer that translates detected part class → target bin coordinates
The LeRobot RobotKinematics solver to convert pixel coordinates into joint-space commands
A state-machine execution loop: detect → pick → lift → move → release → return to home
Dashboard:
Streamlit app showing the live annotated camera feed, per-bin counters, classification confidence, and throughput metrics
Reliability engineering:
Pre-recorded video of a perfect run as a backup demo
A "simulation mode" that runs the full perception → decision pipeline on a laptop webcam if the arm failed
Everything runs on-device to avoid Wi-Fi dependency during judging
Challenges we ran into
The camera-to-robot coordinate transform. Mapping pixel coordinates from the camera to physical pick positions was the hardest part. Small calibration errors meant the gripper would miss the part by centimeters. We solved it by calibrating with a known reference grid and validating pick positions with test runs before each demo.
Lighting sensitivity. The YOLO model performed well under our training lighting but degraded under the venue's harsh overhead lights. We mitigated this by augmenting our dataset with brightness and contrast variations and by controlling the demo workspace lighting as much as possible.
Gripper reliability on small parts. The parallel jaw gripper occasionally failed to grasp flat or thin parts cleanly. We switched to slightly thicker, higher-contrast demo objects and tuned the gripper closing force—prioritizing a reliable demo over handling every edge case.
Time-boxing the scope. The temptation to add multi-part grasp planning, bin-picking from clutter, or defect detection was strong. We forced ourselves to lock the demo to three distinct part classes and a flat workspace, which let us ship a working end-to-end system instead of a half-finished ambitious one.
Accomplishments that we're proud of
A complete perception-action loop working end-to-end in 24 hours — camera → detection → kinematics → arm motion → sorted bin. Many robotics hackathon projects stop at simulation; we shipped a physical, working demo.
A modular pipeline that generalizes. Swapping in a new part type only requires re-training the vision model—the control and dashboard layers stay untouched.
A reliable demo under pressure. With backup video, simulation mode, and on-device execution, we could demonstrate the core value even if hardware or connectivity failed.
A working dashboard that quantifies impact — judges could see accuracy, throughput, and per-bin counts live, not just a robot moving.
Built entirely on open-source and low-cost hardware, proving that flexible industrial automation is accessible to small manufacturers, not just large enterprises.
What we learned
Reliability beats ambition in a 24-hour build. A slow but repeatable pick-and-place demo wins over a fast but flaky one. We learned to design for the demo first and optimize later.
Calibration is half the robotics problem. The AI model was rarely the bottleneck—coordinate transforms, lighting, and gripper mechanics were. Real-world robotics is as much about physical setup as it is about models.
Small datasets go a long way. A fine-tuned YOLO model on ~150 images performed well enough for a controlled demo, which taught us how accessible vision-guided automation has become.
Separation of concerns saves hackathons. Building perception, decision, and control as independent modules meant a failure in one layer didn't take down the whole system—and let team members work in parallel.
The pitch is part of the product. Spending the last two hours rehearsing the demo narrative and preparing fallbacks was as valuable as any code we wrote.
What's next for SortMaster:See. Pick. Sort. Automate.
Handle cluttered bin picking — move from flat, separated parts to randomly piled parts using 6-DoF grasp pose estimation (e.g., GraspNet or AnyGrasp) and depth cameras.
Add defect detection — extend the vision model from classification to anomaly detection so SortMaster can route defective parts to a reject bin, turning it into an inline quality-control station.
Multi-arm and conveyor integration — deploy multiple cobots on a conveyor belt for continuous, high-throughput sorting in real production lines.
Self-supervised learning on the factory floor — let the system improve its own dataset by flagging low-confidence detections for human labeling, reducing the cost of adapting to new parts.
Edge deployment and MES integration — run inference on an edge device (Jetson Nano/Orin) and push sorting analytics to a Manufacturing Execution System for plant-wide visibility.
Pilot with a local MSME — partner with a small-scale manufacturer in Gwalior or the broader MP region to validate SortMaster on real production parts and measure true accuracy and ROI.
Log in or sign up for Devpost to join the conversation.