After learning that Network Rail spends 640 man hours every year to manually review 32,000km of rail track video footage for overgrown vegetation, we thought it must be possible to automate this and make the process more efficient for the staff as well as improve safety and service performance.
We suspected that using computer vision and some modelling we can work out the necessary depth perception to identify hazardous vegetation!
What it does
Veggie on Rails automates vegetation management by detecting hazardous vegetation (defined by the Network Rail standards as appearing over 45 degree angle drawn from the tracks) and flags these to allow the user (NR technician) quickly identify these, without the need to look through all video footage, saving a lot of resource.
It is a massive time saving of valuable human hours that can be freed up for more creative work. It is very cheap -- video footage is already available -- and the neccessary tech is open-source.
How we built it
The solution builds a 3D model of a provided video footage file using OpenSFM, this allows to identify the location of objects in the 3D space and draw the required 45 degree angle to identified objects and flag them as hazardous if they exceed it.
Using computer vision, 3D modelling of distances in the video, we can locate overgrown vegetation.
Challenges we ran into
The biggest challenge in identifying hazardous vegetation is determining the 45 degree angle from the railway track, as the Network Rail standards specify that any vegetation beyond this line is at risk of obstructing the railway. Identifying this angle in video footage is a challenging task for humans too. However, the 3D modelling of the video and resulting coordinates of each identified point, allowed to do this accurately.
Accomplishments that we're proud of
Finding a solution to the 45 degree angle!! It was mind boggling on how to solve it for many hours, but the idea to apply 3D modelling which would output coordinates of identified objects and the direction of the train (camera) was a breakthrough.
What we learned
This was the first experience for the entire team working on a computer vision project of this size. In particular, the use of Structure From Motion to 3D model a video footage. We more than refreshed our geometry knowledge and skills!
What's next for Veggie on Rails
The more we talked with mentors and railway experts during the event the more applications for this solution we found. Some of these include:
- Matching the hazardous points with GPS data and mapping these out
- Using artificial intelligence and object detection to find obscured signs
- Use AI to identify obstructed infrastructure: signs, signals, rail side buildings
- Determine vegetation growth rate
- Difference analysis over time (if GPS data provided)
- GPS Smoothing problem can also be solved with this
- The Deutche Bahn challenge of identifying debris is a straightforward extension
- Running the code on optimised hardware would considerably improve the efficiency of the software. The desire will be that the processing will be able to run faster than real time.