AutoEye

AutoEye is an AI-powered bridge that scans vehicles using 360° cameras and LiDAR to detect damage or defects instantly — revolutionizing pitstops, manufacturing, and quality inspections.

Comment

Inspiration

In motorsports and advanced manufacturing, precision is everything. A single undetected crack, scratch, or misalignment can result in catastrophic failure, performance loss, or high repair costs. We were inspired by how Formula 1 pitstops achieve incredible efficiency — yet rely heavily on human visual inspection. In both racing and production lines, human inspection introduces delays, inconsistency, and subjectivity. This inspired AutoEye — a vision-based, AI-powered inspection system designed to automatically detect exterior damages, missing components, and anomalies in vehicles or machinery, enabling faster and more accurate quality control.

What it does

AutoEye is a conceptual design for an AI-enabled visual inspection bridge that uses multi-angle cameras, depth sensors, and cloud-based analytics to detect exterior changes and defects automatically. When deployed, a vehicle or product would pass through the bridge structure, where: Cameras and LiDAR sensors capture synchronized images and 3D data from multiple angles. The images are processed on an edge device and transmitted to a cloud-based AI inference pipeline (Azure Machine Learning). The system compares current visuals with baseline images or CAD references using structural similarity algorithms and deep learning models. Anomalies such as dents, cracks, paint inconsistencies, or component displacement are detected and highlighted for operators via a dashboard interface. The same framework could be adapted for:

  1. Manufacturing: Real-time defect detection during production.
  2. Motorsport: Instant pitstop inspections for damage assessment.
  3. Logistics: Entry/exit inspections for vehicle or shipment integrity.

How we built it

We developed a comprehensive system architecture, selected the technology stack, and created a working model pipeline using Python simulations and Azure services. Our conceptual architecture consists of :

  1. Hardware Design:
  2. Modular bridge frame with multi-angle RGB cameras and LiDAR sensors.
  3. Adjustable LED illumination for consistent lighting.
  4. Edge inference hardware (NVIDIA Jetson / Raspberry Pi) for low-latency processing.
  5. Software Design:
  6. Preprocessing: OpenCV, Pillow, NumPy for image enhancement and normalization.
  7. Anomaly Detection: Structural Similarity Index (SSIM) and YOLOv8-based visual difference model.
  8. 3D Mapping: Open3D and pyrealsense2 for geometric surface analysis.
  9. Backend & Visualization: FastAPI for inference serving; Power BI for data visualization.

Azure Cloud Integration: Azure Machine Learning -> Model training and deployment pipeline Azure Blob Storage -> Image and result data storage Azure IoT Hub / IoT Edge -> Real-time communication between devices Azure Functions -> Automated workflows and triggers Azure Cosmos DB -> Metadata and inspection log storage Power BI / Data Explorer -> Reporting and dashboard analytics

The Algorithm : The AutoEye system uses structural similarity (SSIM) to measure pixel-level differences between the baseline and current frame. Regions below a certain SSIM threshold are flagged as anomalies. These are then classified using deep learning models such as YOLOv8 or Detectron2.

Challenges we ran into

  1. Designing a multi-camera layout that ensures complete 360° coverage without occlusion.
  2. Balancing lighting consistency for reflective surfaces.
  3. Creating a robust training dataset for damage and defect recognition.
  4. Managing real-time data flow between edge devices and the Azure cloud.
  5. Developing a workflow that can scale to different use cases (motorsport, industrial, logistics).

Accomplishments that we're proud of

  1. Developed a detailed system architecture and pipeline combining AI, IoT, and cloud computing.
  2. Designed a scalable inspection framework adaptable across industries.
  3. Created a simulation-based validation pipeline for image differencing and object detection.
  4. Identified and integrated the appropriate Azure ecosystem components for deployment.
  5. Built a clear roadmap for hardware prototyping and dataset development in future phases.

What we learned

  1. How to architect an AI-driven visual inspection system from the ground up.
  2. Practical understanding of computer vision, object detection, and image differencing techniques.
  3. Integration of Azure cloud services for ML deployment and IoT data flow.
  4. The significance of edge–cloud hybrid systems for real-time industrial AI applications.
  5. Challenges in achieving accuracy, scalability, and reliability in industrial AI designs.

What's next for AutoEye

  1. Prototype Development: Build a scaled-down physical bridge with multiple cameras and an embedded processing unit.
  2. Integration of thermal and infrared sensors to detect leaks and overheating components.
  3. Implementation of AR-based visualization for real-time defect overlays using HoloLens or tablets.
  4. Addition of predictive analytics and digital twins for failure forecasting and maintenance planning.
  5. Development of a lightweight mobile version for on-field or drone-based inspections.
  6. Expansion to federated learning architecture for continuous model improvement across distributed AutoEye systems.

Built With

Share this project:

Updates