Machine vision is a really interesting technology, with use cases that span from business services, to improving the life of a person using IOT by integrating into their smart home, to social good by tracking a the cars passing by or who has been speeding past your house, this project offered a unique challenge that and opportunity to build a core founding framework and then create small uses that showed its flexibility and usability. Furthermore the idea of one day being able to report people speeding by your house by suppling a timestamp, license plate, video and speed is a great way to help make neighborhoods a safer place.
What it does
Plate-sniper at its core is just a way to isolate a license plate inside an image or a video. however we thought just doing that would be too easy, so we created three sub-projects:
The Business Use Case: AutoNation asked and we delivered. Not only do we have a demo on our website of our codebase checking to see if a user is linked to a user profile based on the license plate that was just read, it sends out a message to the staff about the arrival of that customer, their name, time of meeting, and whom they are meeting! look for that in the Business Services tab on our webpage!
The Home Service: What is cooler than your door recognizing you when you walk up to it? your house recognizing your license plate when you drive into your driveway! We implemented a quick service that would allow a user to be recognized and their settings activated. it is ready to plug and play with a smart hub/ other smart device controller like Alexa. Unfortunately due to not owning any smart devices we could not actually implement this, so instead you will see the settings for a validated user in the Home Services Tab
The Social Service: Finally we have a social good component. we actively have an logged list of all of the vehicles that have been recorded along with a timestamp. The image is also stored for later use. This service was also supposed to add in speed and distance projection to track for and notify of people speeding. Unfortunately we were only able to calculate distance moved and didnt have enough time to build out the speed tracking functionality. The License plate List can be seen in the License List tab.
How I built it
Currently our solution utilizes Python and its implementations of OpenCV and Tensorflow for extracting licenses from images. This is then uploaded to Google’s Cloud Vision API for OCR (Optical Character Recognition). This is done because we could see an increase in accuracy from Googles cloud vision when this was done. After acquiring the license plate text, this is uploaded to a Google SQL MySQL database. This database also holds the message text for the home automation portion. The front end consists of a Microsoft Azure Static Web app utilizing Bootstrap and Node.js which pulls its data from the MySQL DB.
Our development tools consisted of PyCharm, Jupyter, Visual Studio Code, and MySQL Workbench. We also used many other Common API’s such as numpy, pandas, matplotlib and keras.
Challenges I ran into
Some challenges we ran into consisted of the following:
- Securely accessing credentials in an Azure Web App
- Accessing a Google DB from a serverless web app
- Finding a nice front end framework that was easy to implement (had to fall back on Bootstrap)
- Creating an Azure container for a MySQL db (fell back on Google MySQL)
Accomplishments that I'm proud of
We are both very proud of the fact we were able to create soo much in such a short time with only two people! we also are happy that the license plate reader works on a plethora of different situations(multiple nations, in the dark, upside down, on other types of vehicles, etc). Finally we are simply proud that we can build something that has a use case for all, specifically as it applies to the social good sector.
What I learned
What's next for Plate-Sniper
The very next thing we want to finalize is reading the speed from the same or a second camera. We were able to implement the distance calculator based on the plate scale at a known camera location, however, we were unable to finalize computing the speed between frames in the video (but we are close!)
This would allow us to present a more feature rich solution for tracking safety on neighborhood streets which was one of our original targets. The frontend has hidden fields to show the speed once it was joined with a plate