-
-
Figure 4. Dashboard Showing Medium Risk Detection
-
Figure 5. Dashboard Showing High Risk Detection
-
Figure 1. Operational Flowchart of the RockSense System
-
Figure 3. Initial Hardware Prototype of the RockSense System
-
-
Figure 2. Complete Hardware Circuit Diagram of the RockSense Prototype
RockSense – AI-Based Rockfall Prediction and Alert System
Inspiration
Rock fall and landslides still result in serious accidents in open pit mining sites, hill roads, and excavations. Current methods used to prevent rock falls employ inspection by hand or monitoring devices that are costly, hence making them not suitable for use in many other sectors. This project aimed at developing an affordable artificial intelligence method to monitor rocks.
What it does
RockSense is a smart edge computing monitoring tool which continuously monitors rock surfaces through a Raspberry Pi camera. The recorded videos are fed into a YOLOv11 deep learning model to detect the cracks. Unlike other methods of detecting cracks, which trigger alerts upon their occurrence, RockSense waits for 15 seconds for verification of cracks to avoid false detections.
After the 15-second verification process, the extracted features such as average confidence, maximum confidence, number of detections, average crack area, and monitoring time are classified by a Random Forest classifier to determine the level of risk (Low, Medium, or High).
In case of detection of high risk, the Arduino-led LED and buzzer alert system are triggered instantly with the display of the crack detected, the heatmap, and the risk level at the monitoring panel. In addition, the worker GPS location tracking module ensures that the workers are always safe.
How we built it
Artificial Intelligence, Edge Computing, IoT, and Embedded Systems have been fused into one safety solution.
- Raspberry Pi 4 with Raspberry Pi Camera Module for live surveillance
- YOLOv11 for live crack detection
- Random Forest for risk assessment
- Python, OpenCV, NumPy, and Scikit-learn for AI processing
- Arduino Uno for LED and buzzer for emergency situations
- GPS module to track workers
- React.jsdashboard with FastAPI backend
Challenges we ran into
Development of RockSense involved overcoming some practical engineering problems.
- Ensuring effective detection of cracks in varying light settings.
- Avoiding false alarms due to the movement of objects temporarily.
- Efficient running of AI inferences on Raspberry Pi hardware.
- Combining AI models with Arduino hardware for instant alerts during emergencies.
- Creating a reliable communication channel from edge devices to monitoring dashboard.
Accomplishments that we're proud of
- Created an AI-based prototype system to detect rockfall events.
- Managed to incorporate Computer Vision, Machine Learning, IoT, and embedded technologies.
- Adopted an innovative approach with a 15 seconds of stabilization feature.
- Created an affordable solution that can be applied to mining sites, quarry sites, tunnels, and landslide-prone areas.
What we learned
This work helped us gain valuable insights about Computer Vision in real time, embedding of AI into edge devices, hardware interfacing, serial communication, and design of intelligent safety systems. Also, we learned how to optimize machine learning models on resource-constrained edge devices.
What's next for RockSense
Our plans for the future include improving RockSense with the use of thermal imaging, LiDAR, seismic and vibration sensors, weather prediction, inspection drones, and Digital Twin technology. Future versions will also allow for cloud analysis, automatic response, and application to mines, highways, tunnels, dams, and any other geological hazardous areas.
Log in or sign up for Devpost to join the conversation.