Inspiration

Construction site safety is critical, but manually checking large numbers of site photos for missing helmets and high-visibility vests can be slow and inconsistent. We wanted to build a system that helps safety officers identify photos that need closer inspection, while keeping humans responsible for the final decision.

What it does

SafeSite Vision is an AI-powered construction safety inspection system that analyzes uploaded site photos for workers and personal protective equipment (PPE). It checks image quality, detects safety gear, and re-examines uncertain detections by zooming into regions of interest and enhancing image contrast.

The system returns an annotated image, inspection status, detection confidence scores, and a step-by-step trace explaining its process. It also saves inspection evidence to AWS S3 and sends an email alert when potential PPE issues are identified. Every inspection remains subject to human review; the system does not certify a site as safe.

How we built it

We divided the work between two contributors and integrated our components into one end-to-end application.

Vision agent: Built the inspection pipeline using Python, OpenCV 5, and a CamThink construction PPE YOLOv8 ONNX model. The agent checks image quality, performs object detection, re-checks uncertain results using zoomed crops and contrast enhancement, and records its decision trace.
Backend and cloud: Built a FastAPI server to receive images, run inspections, return structured results, and connect the application to AWS services.
Web interface: Developed a browser-based interface for uploading photos and viewing annotated detections, confidence scores, inspection status, and the agent's reasoning trace.
Cloud integration: Used Docker and AWS EC2 for deployment, Amazon S3 to store original images and inspection evidence, and Amazon SNS to send email alerts. IAM permissions help restrict cloud access.

We integrated these components so a user can upload a photo, receive an inspection result, and review the evidence through a single workflow.

Challenges we ran into

One challenge was handling uncertain detections without blindly trusting the model. We implemented additional checks using zoomed image crops and contrast enhancement, while preserving the original detections for comparison.

We also encountered false alarms when the model detected missing equipment that was not required by the site's policy. We addressed this by making the required PPE configurable, with helmets and high-visibility vests as the default requirements.

Another challenge was connecting the vision pipeline to the backend, web interface, and cloud services while keeping the system usable when cloud storage or alerts are unavailable. We designed the application so an inspection result can still be returned even if saving evidence fails.

Accomplishments that we're proud of

Built an end-to-end application connecting image upload, AI inspection, cloud storage, and email alerts.
Developed a vision agent that checks image quality and re-examines uncertain detections instead of relying on a single model pass.
Created an interface that makes the model's findings and reasoning trace visible to users.
Deployed the application using Docker and AWS services.
Kept human review central to the workflow, with explicit statuses and evidence to support safety officers.

What we learned

We learned how to integrate computer vision, backend APIs, frontend development, and cloud infrastructure into a single application. We gained practical experience with ONNX inference through OpenCV, image enhancement, confidence-based decision-making, REST APIs, Docker, and AWS services.

We also learned the importance of communicating uncertainty honestly. A detection model can make mistakes, especially when equipment is hidden or difficult to see, so the system must support human judgment rather than replace it.

What's next for SafeSite Vision

Our next steps are to evaluate the system on a larger collection of representative construction-site photos, measure its accuracy and false-alarm rate against manually labelled results, improve difficult and partially occluded detections, and expand support for different site-specific PPE policies. We also plan to explore video or live-camera inspection in the future.

SafeSite Vision is designed to help people make better-informed safety decisions—not to replace safety professionals.