Drone Infrastructure Research Technology

The OpenMV camera

Inspiration

A recent hurricane hit with such force that it brought multiple bridges to barrier islands down during the store. Leaving the people who live there with little escape. It just highlights how critical our infrastructure is as it could be life or death. Being able to predict when someone might happen and quantify the number of fractures on a structure could help engineers make the necessary repairs before a serious failure occurs.

What it does

We used an Openmv h7 camera to classify surface fractures on buildings, bridges, and overpasses. Once we get a positive classification, the location and other metadata are uploaded via MQTT(rabbitmq) to Adafruit IO. This is then used to populate our frontend dashboard.

How we built it

We used Edge Impulse to do transfer learning using MobileNet and two other datasets. One dataset had the structural cracks and the other was a general image dataset to decrease the false positive rate. This model was then put on the openmv camera to be ready for deployment. I created an MQTT server with rabbitmq to upload the data to Adafruit IO.

We created the frontend using Dash, a python based web framework primarily used for creating dashboards. We also used bootstrap to add some styles to our dashboard.

Challenges we ran into

The majority of our team was new to hackathons, so we had to learn new skills and adapt to the environment as we went along in the project. However, we were able to collaborate with each other and overcome anything that stood in our path!

The hardware was having some issues as well due to difficulties when doing an HTTP POST from the camera as it would crash if it got any other error other than a 200 error.

Accomplishments that we're proud of

We Finished!! Also, we are proud of our teamwork skills, frontend development, and the accuracy of our deep learning model.

What we learned

Many of our members were able to further develop their python programming and frontend development skills by learning new technologies such as dash and bootstrap. We also learned more about how hackathons work which will be beneficial to our less experienced members as they continue along in their educational and professional careers.