Inspiration
Scanning a large space with multiple cameras creates a coordination problem: operators can scan the same area repeatedly while leaving other areas unexplored. We wanted to turn visual evidence into a simple instruction: "This area is scanned. Move ahead."
What it does
Swarm-coverage combines stereo-camera observations into a shared 3D map and helps operators decide where to scan next. It checks possible overlaps, marks areas as covered when the evidence is strong, and sends move-on notifications. Weak matches go through an approval interface, while poor tracking triggers warnings. A dashboard displays camera views, the map, a 3D point cloud and the decision history, with an upload workflow for recorded stereo footage.
How we built it
We used OpenCV 5 for stereo rectification, feature extraction, tracking-quality checks and overlap verification using ORB and MAGSAC. RTAB-Map estimates camera motion, and Swarm-SLAM combines keyframes into a shared pose graph. Our coordinator supports rules and Amazon Nova 2 Lite through AWS Bedrock. In the Nova-driven EuRoC tests, visual evidence influenced tool calls to mark coverage, notify operators or request approval. Code-level guardrails prevent low-confidence overlaps from being automatically accepted. The application runs on AWS EC2 with ROS 2, FastAPI and SQLite.
Challenges we ran into
Repeated structures can look like the same location and cause incorrect map merges. We also had to handle stereo calibration, tracking loss and coordination between several camera streams. Limited compute on our 2-vCPU server caused processing to fall behind normal-speed input. These challenges made careful validation and visible failure handling essential.
Accomplishments that we're proud of
We connected stereo processing, multi-camera mapping, 3D reconstruction and operator guidance in one working pipeline. Our recorded EuRoC tests provide examples of OpenCV evidence changing the agent's next action. We also built browser-based upload jobs and generated a point cloud of approximately 595,000 points from the AirMuseum test footage. Evaluation includes failure cases, rather than only successful demonstrations.
What we learned
A useful vision agent must do more than describe an image: its observations should change what happens next. We learned to connect decisions to measurable visual evidence, enforce safety checks in code and make uncertain cases visible for review. We also learned that successful map generation does not automatically establish reconstruction accuracy or real-time performance.
What's next for Swarm-coverage
We plan to evaluate our wearable-camera recordings, improve rejection of look-alike locations and test larger compute configurations. We also want to add live camera streaming and measure end-to-end latency, with the goal of making coordinated 3D scanning faster and more reliable.
Log in or sign up for Devpost to join the conversation.