One of the most significant factors related to driving ability is the condition of the drivers itself. Be it sleep deprivation, alcohol or drugs, unfit drivers should not be getting behind the wheel. We decided to use this weekend and our passion for technology to develop a web app that would keep impaired drivers locked out of their car, and permit suitable drivers to take the wheel. We hope to use this app to enhance Toronto as a smart city, and keep drivers as well as everyone around them safe.
What it does
Our app captures the face of a potential driver and using machine learning, evaluates whether the driver is fit to be behind the wheel. Our dataset was comprised of drowsy faces, and faces under the influence of impairing substances. If the driver is unsuited to drive, the app will not unlock the car, but if the driver is assessed to be fit, the car will unlock.
How we built it
Using Heroku, we deployed a web/mobile application with a Flask backend that captures a photo that communicates with Azure's cloud services. Not only is the photo uploaded automatically, but also any changes to the code are deployed on a running basis. Next, the image is taken from Azure's cloud and is analyzed by a machine learning model developed with Google Cloud Platform (GCP). The result is set to the Smartcar API that has the capabilities to unlock/lock the car.
Challenges we ran into
Initially, we wanted to create an exclusively mobile app that can interface with Azure to send images. However, we ran into complications merging the two platforms and these challenges drove us to making the app web and mobile compatible through the use of Heroku. While interfacing Azure with Flask did provide some challenges at first, we resolved these obstacles with the fantastic Microsoft mentors (special thanks to Aaron Wislang!). Assembling enough data to properly train a good face recognition model also proved to be a challenge due to the lack of time and data.
Accomplishments that we're proud of
We're proud of the fact that we were able to train a face recognition model to assess suitable drivers in a short period of time. Also, the hands-free means of integrating all of our operations meaningfully onto the cloud. Lastly, we're excited to having had the opportunity to successfully have developed a software/hardware interface with the car.
What we learned
We learned a lot of technical skills and tools, namely Azure Cloud Services, Smartcar API and building web apps for Flask.
What's next for LockPic
Continue training the model and improving it's accuracy. In the future, we hope we can integrate continuous face tracking of drivers while on the road and issue warnings through the car or hardware devices if their alertness appears to change.