Keeping in line with the theme of MHacks X and as a tribute to the birthplace of the Automotive Industry, we decided to focus on one of the most important but often neglected aspects of motor travel - Roads. We decided to come up with a solution that would help all the stakeholders involved with them - Commuters/Pedestrians, Automobile Manufacturers and the City.
What it does
We decided to build an Augmented Reality HUD which would use a binary image classifier to identify and highlight hazards on the road - Potholes, Asphalt cracks, Branches, etc whilst also cataloging the location of said hazard in a central server to alert other drivers, the city, and pedestrians. The Hazards could be anything from a fallen tree, potholes, cracked Asphalt, or an Accident prone zone (identified by trends) and point out faults that can be fixed by the City to them (road damage, poorly placed or lit signs, etc.).
How we built it
Trying to use new and interesting technology, we have implemented the app in React Native using Expo. The backend is deployed on Google Cloud Platform, with a node.js REST API and a mongodb database for storing locations of potholes. Google Maps was a crucial part of the app and its easy tutorials helped to make our project go faster.
Challenges we ran into
We had issues stemming from running multiple fragments at the same time in the Expo environment. Streaming Live video and still images from the device to our API endpoints with Geolocation data was challenging but worth the result. Finding data to train our Binary image classifier required some exceptional Google-Fu as this isn't a widely explored area of research.
Accomplishments that we're proud of
Producing a Livestreaming application which captures high-quality images and geolocation data with negligible latency that classifies road images in real-time and simultaneously builds a network of quantified roads (as per aforementioned criterion).
What we learned
Google Cloud Platform, Expo.io and Cloud ML Engine
What's next for Paths
- Integrate better performant CV algorithms to make the network roundtrip to identify more road hazards in an efficient manner.
- Consolidating existing observations as a map layer (Google or OSM) to identify sections of the road in the city that require urgent attention, produce best possible paths for ride quality (which could be perused by driving enthusiasts, manufacturers to show off their vehicles to prospective buyers),
- Possibly improve on-device detection to sparingly use the network for most workloads
- Talk to manufacturers/insurers about the importance of educating customers about the need to choose better routes for vehicle longevity, ride quality and safety.