Hazard Detection System

Inspiration

Growing scooter and micromobility use on campus creates safety risks for both riders and pedestrians. Riders face unexpected hazards like potholes and curbs, while pedestrians aren't always aware of approaching vehicles, motivating a proactive, real-time alert system.

What It Does

A real-time hazard detection tool for scooters using a YOLO-based object detection model. It identifies physical and environmental hazards in the rider's path and delivers audio alerts for quick reaction, minimizing unnecessary notifications by focusing only on in-path dangers.

How We Built It

Four components work together:

YOLO object detection: pretrained models augmented with custom-trained models for potholes, rocks, stairs, and speed bumps
Path detection (MobileSAM): identifies the scooter's projected path to filter out off-path objects
Hazard assessment: evaluates detected objects by size, position, and proximity
Alert system: triggers audio alerts and logs detections to CSV with timestamps

About the team:

Toby - Created the core hazard detection logic (deciding when an object is actually a threat based on size, position, and proximity) and merged the project into the final web app
Ashwin - Identified what YOLOv8 can detect out of the box and trained custom models for hazards it couldn't handle (potholes, rocks, stairs, curbs, and speed bumps), sourcing public datasets from Roboflow and collecting custom data
Kenny - Built the path detection system using MobileSAM to segment the road surface in real time, so the app only alerts on objects that are actually in the scooter's path rather than off to the side
Katherine - Created the alert system: audio feedback when a hazard is detected, a 2-second cooldown to avoid constant beeping, and CSV logging of every detection with timestamps

Challenges

Finding quality datasets for roadway hazards was difficult, as most were small or poorly labeled. Training on Apple Silicon was slow, and combining multiple models introduced misclassification issues. We iterated extensively on confidence thresholds and configurations, and later moved training to Google Colab for GPU access.

Accomplishments

Successfully integrated YOLO, OpenCV, and MobileSAM into a single real-time system running on a standard webcam. Achieved a balance between detection accuracy and usability, where alerts are meaningful without being overwhelming.

What We Learned

How to train and fine-tune object detection models for dynamic real-world environments
The importance of user-centered design in safety systems, especially for alert mechanisms

What's Next

Transition from software prototype to a portable hardware solution, specifically a compact camera and mini-computer mounted directly on a scooter. Future work includes edge device optimization, a practical mounting system, and improved usability for everyday deployment.