Pedestrian fatalities were at a 25 year high in 2017. One of the main causes? Lack of awareness. Most efforts are focused on reducing accidents from the driver's end, but rarely do people consider the perspective of the pedestrians. By having safety precautions on both ends, pedestrian fatalities would significantly decrease.
What it does
The program uses a Machine Learning model trained using Microsoft Azure that notifies the user when it's safe to stop or go at a crosswalk to decrease pedestrian fatalities in a live camera feed.
How we built it
We used Microsoft CustomVision to train an image classifier to identify whether a photo/video captured a "Go" or "Stop" pedestrian crosswalk signal. If the pedestrian is not at a crosswalk, the classifier outputs "N/A". The ML model was exported as a CoreML file and implemented into an iOS app based on Swift/Xcode. The live camera feed was implemented using the AVFoundation library to create our product.
Challenges we ran into
- Choosing the right classifier
- Training our classifier to maintain highest precision/recall
- Converting video into an image sequence
- Exporting the CoreML file, ensuring that it had the right outputs
- Linking the Phone camera to the app
Accomplishments that we're proud of
- Learned how to develop an iOS app in a night!
- Learned how to use Microsoft Azure
- Achieving 87% Precision and Recall in our Image Classifier
- Having a working product
- Having a cool team name and product name
What we learned
- Phone app development
- iOS development
- Microsoft Azure
- Sleep is important
- Tagging images is very tedious
- Caffeine is a must
What's next for Crossy Road
- implement the software into glasses, smart watches, or other personal portable devices so the user doesn’t have to bring their phone all the time and can safely text while walking
- Implement audio feedback
- Collect more data to increase accuracy
- More user-friendly and interactive UI