SafeSense

Cool Side angle of drone
Back angle of drone
dashboard example

Inspiration

Infrastructure inspection is still largely manual, time-consuming, and expensive. Workers must physically inspect buildings, sidewalks, and structures, which introduces safety risks and delays construction timelines. We were inspired to explore how drones and AI could automate this process—capturing critical data from hard-to-reach areas while reducing human effort. With construction costs rising across Canada, we wanted to build a system where inspections are faster, safer, and more scalable.

What it does

SafeSense is an AI-powered drone inspection system that captures key frames from a live video feed, detects structural damage such as cracks or surface degradation, and generates professional inspection reports in real time. Instead of analyzing every frame, the system intelligently samples frames to reduce unnecessary computation while still identifying meaningful changes in the environment.

The system then uses AI to transform detection results into structured, easy-to-understand reports with actionable insights, helping homeowners, contractors, and inspectors make faster decisions.

How we built it

We built SafeSense as a full pipeline:

A Raspberry Pi with a webcam captures live video footage
Frames are streamed over a socket connection to a local server
The server processes incoming frames and selectively analyzes them
Damage detection is performed using computer vision techniques
AI (Gemma) generates structured inspection reports based on detected issues
A Flask-based web dashboard displays the live feed, captured frames, and downloadable reports

We also optimized performance by only analyzing frames periodically or when changes are detected, reducing latency and improving efficiency.

Challenges we ran into

One of the biggest challenges was setting up reliable communication between devices. We had to debug socket connections across different machines, resolve network issues (like incorrect IP routing), and ensure frames were transmitted efficiently.

Another challenge was camera integration. We initially encountered issues with OpenCV backends and device access, especially when switching between OBS virtual cameras and a Raspberry Pi webcam. Fixing backend compatibility and ensuring stable frame capture took significant debugging.

We also had to carefully design our system to avoid processing every frame, which would be computationally expensive and unnecessary. Implementing efficient frame sampling was key to making the system practical.

What we learned

We learned how to build an end-to-end real-time system that integrates hardware, networking, computer vision, and AI. This included working with sockets for streaming data, handling concurrency with threads, and optimizing performance in a live pipeline.

We also gained experience using AI models to generate meaningful outputs from raw data, turning simple detections into structured, professional reports.

What’s next

In the future, we would expand SafeSense by integrating more advanced damage detection models, supporting multiple drones, and deploying the system to the cloud for large-scale infrastructure monitoring. We also see potential in integrating real-time alerts and predictive maintenance features.