Arc - AI-Powered Wall Scanner

🎯 Inspiration

The inspiration came from watching contractors spend hours measuring rooms and manually documenting wall elements like outlets, switches, and fixtures. Traditional 3D scanning requires expensive LiDAR equipment, while manual measurements are time-consuming and error-prone.

What if we could turn any smartphone into an intelligent wall scanner?

🧠 What We Learned

Computer Vision at Scale

• Trained custom YOLO models on 5,002 hand-labeled images across 5 wall element classes
• Learned the challenges of real-time object detection with confidence thresholding
• Discovered the importance of data quality over quantity in specialized domains

3D Reconstruction Pipeline

• Depth estimation using Intel's DPT (Dense Prediction Transformer)
• Point cloud processing and mesh generation with Open3D
• ICP (Iterative Closest Point) registration for multi-wall stitching

System Architecture

• Built dual detection system combining specialized and general models
• Implemented real-time processing with frame stabilization
• Created modular pipeline: Detection → Depth → Point Cloud → 3D Model

🔨 How We Built It

1. Data Collection & Training

# Custom dataset: 5,002 images across 5 classes
classes = ['mirror', 'thermostat', 'vent', 'wall_socket', 'window_box']
train_split = 4,001 images (80%)
val_split = 1,001 images (20%)

Built With

• html/css
• javascript
• numpy
• opencvscikit-learn
• python
• pytourch
• ultralytics