Caterpillar Inc., a sponsor of the BuiltWorld Hackathon 2019, proposed the problem of not being able to effectively analyze the usage and wear of their machineries across construction work sites.
As an effort to approach this problem, we decided to take advantage of the already-available surveillance video feeds and build a machine learning model on top of the available data set. We collected a considerable amount of Caterpillar work site videos and built out the machine learning model using Mask R-CNN.
Initially, we were using YOLOv2 to do so, but we encountered multiple memory and storage problems with the EC2 instances we had on AWS, so we decided to switch to Mask R-CNN as time turned into a big concern (training, building, and etc.). Fortunately, at the end, we are able to recognize the number of scoops of particular machines and their predicted time for maintenance and repair.
In order to present the complex data that we collected and analyzed, we created a intuitive online dashboard for users to view the data in real time. We made sure that the online interface is intuitive, user-friendly, and mobile-friendly, so that even workers on the construction site can access it easily and see the information quickly.