Inspiration
Construction safety is still largely managed by manual spot checking, despite the fact that the threat is always present and the consequences of failure are severe. We created OSHA-Vision because we wanted to bridge the gap between scheduled inspections by leveraging ordinary video present on the construction site as an efficient tool for automated construction site safety auditing. The basic premise was simple: if video is already being recorded on the construction site, that video should be able to detect potential hazards before they become incidents.
What it does
The application, named OSHA-Vision, allows the user to upload the video from the construction site and analyze the video to identify safety issues related to OSHA regulations. It takes samples from the video, uses AI vision to identify issues such as the lack of hard hats, lack of safety vests, fall hazards, ladder hazards, scaffolding hazards, electrical hazards, housekeeping hazards, and so on, and then groups these repeated issues into incidents. It also provides a weighted risk score from 0 to 100 and produces an OSHA-compliant safety report.
How we built it
We created the product as a full-stack pipeline. Our frontend was created with Next.js and manages upload, processing status, and results view. Our backend was created with FastAPI and manages upload, video metadata storage, and background processing initiation. Supabase was used for data storage and video/file storage of uploaded videos and evidence frames. In the worker pipeline, we use OpenCV to pull frames from uploaded videos, Claude Vision to analyze sampled frames of video for OSHA violations, a rule/citation engine to map findings to specific OSHA standards, a risk engine to score severity of violations, and a report generator to create a structured compliance report from raw detection results. We have added a local PPE detection path in order to still detect basic hard hat and vest violations if the external vision engine was unavailable.
Challenges we ran into
The hardest part was making the unstructured video data presentable for compliance review. Construction video is messy, with many different camera angles, people crossing the screen, varying visibility, and the same violation appearing across multiple video frames. We had to balance coverage, cost, and speed by not reviewing every frame of video, as well as remove redundant violations so that the output was useful. The other challenge was taking the output of the artificial intelligence, making sure it was structured so that it could be scored, stored, and turned into a report, as opposed to being vague.
Accomplishments that we're proud of
We delivered a complete end-to-end working product, not just a demo screen. A user can upload a video, analyze it, get timestamped violations, view evidence frames for violations, see a normalized risk score, and read an OSHA-report-style summary all in one shot. We're also happy to say that our system extends beyond just PPE and is organized along broader OSHA construction categories, so it feels like the beginnings of a real safety system instead of a computer vision experiment.
What we learned
We learned that the hard part of AI safety tooling isn't just about accuracy. It's about system design: selecting the appropriate sampling strategy, transforming noisy model output into structured incident data, determining the weights in a way that produces a useful score, and providing fallbacks to ensure the system still works if one dependency fails. We learned that adding vision analysis to standards citations and natural language reporting makes the results much more actionable to a real user.
What's next for OSHA-Vision
The next step is to move from video audits to live monitoring. This means we want to move from the ability to upload video audits to the ability to have live video feeds, instantaneous hazard detection, and site-level risk tracking in real-time rather than post-hoc analysis. Additionally, we want to increase the reliability of our hazard detection capabilities through more specialized vision models, worker- and zone-level context, and deeper regulation mapping to tie every finding to the most relevant regulation to the jobsite. Long term, the dream is to have safety auditing be continuous, not just something that is done periodically, and to have it be something that is done proactively and not just reactively.
Log in or sign up for Devpost to join the conversation.