BoxBeacon

Our robot scans shelves using ArUco tags and AI vision to find user-specified cases. If found, it points with a torchlight; if not, it “shakes head” to signal absence. Smart, simple, shelf-savvy

Comment

Inspiration

💡 Inspiration

Our inspiration came directly from the Amazon sponsors, who shared insider insights into their distribution centers. Hearing firsthand about the challenges faced by workers and the limitations of current automation made us realize the real-world impact a solution like ours could have. Their stories motivated us to focus on helping robots visually locate and identify specific packages—just like a human would.

📦 What it does

BoxBeacon is a robotic system that scans shelves using an RGB camera to detect and identify user-specified cases. It first searches for ArUco markers to locate known boxes. If no marker is found, it uses a YOLOe-based model to segment the shelf and detect case boundaries. Using Hough Line detection and camera geometry, it estimates the physical dimensions of visible cases and compares them to a database to find the desired one. If the case is found, the robot points a gripper-mounted torchlight at it. If not, it moves the gripper side-to-side to indicate absence—like shaking its head.

🔧 How we built it

We started with a Hiwonder ArmPi Pro robot kit: an omnidirectional wheeled cart with a 5-DoF robotic arm and an RGB camera attached near the gripper.

The main struggle was bridging our ROS1-based robot with our ROS2-enabled laptop, which we used for running the segmentation model due to its GPU. We used ROS bridges to relay camera streams and commands. On the vision side, we implemented a pipeline that first checks for ArUco markers, then falls back to segmentation and Hough Line detection to extract case corners and estimate dimensions.

🧱 Challenges we ran into

  • ROS1–ROS2 Integration: Making both systems communicate took considerable time and debugging.
  • Segmentation Robustness: Lighting and angles affected segmentation quality.
  • Hough Line Edge Detection: Extracting clear lines from noisy scenes proved tricky.
  • Hardware Glitches: We spent hours debugging what turned out to be a loose camera wire!

🏆 Accomplishments that we're proud of

  • Successfully ran real-time segmentation and geometry extraction on a moving robot.
  • Built a working ROS bridge between ROS1 and ROS2 environments.
  • Designed a user-intuitive robot response (torchlight pointing and "head shake") that mimics human-like feedback.
  • Tackled (at least attempted to) a real-world problem inspired directly by industry input.

📚 What we learned

  • Real-world robotics is messier than simulation—hardware will humble you.
  • Lighting and camera reliability are critical in perception tasks.
  • Learning-based vision models are far more robust under environmental changes.
  • Multi-ROS communication is tough but doable—and worth it!

🚀 What's next for BoxBeacon

If we had more time, we’d complete the full 3D inference pipeline to estimate dimensions from detected case corners and improve the system’s autonomy. We also envision deploying this system on larger robot platforms with multiple sensing modalities for more complex warehouse environments. Our goal: a plug-and-play shelf-scanning assistant for the modern distribution center.

Built With

Share this project:

Updates