Inspiration and What it does

The inspiration for this project stemmed from the passion of leveraging emerging technologies such as Machine Learning and Artificial Intelligence in real-world applications. As such, a use case was explored where a cloud-based infrastructure on the Google Cloud Platform could be integrated at AutoNation to provide a personalized customer experience specifically in the service center department by extracting the text from the license plate of a car and retrieving relevant customer information from a database using the value of the license plate as a lookup criterion. Key information such as scheduled appointments, the reason for visit, and customer contact information could be used by a service advisor at AutoNation to speed-up the check-in process and promote a high-quality customer-centric environment.

How we built it

pyVision was built with several tools and technologies, including a significant component of cloud architecture from the Google Cloud Platform. To enable connectivity and custom functionality pertaining to this specific use case, we created a Python extension that would automatically extract and solely return the text for a particular license plate. Additionally, there is further functionality to save the results directly as a .txt file into our Google Cloud Storage component. A proof-of-concept Cloud SQL database was created and populated with customer data mimicking the AutoNation environment. The last element is the graphical user interface or GUI, which was built with Java/JavaFX and automatically populated the GUI's informational dashboard utilizing querying via the extracted text as the filtering criterion.

Challenges we ran into

While we did not need to create our own model for text extraction, leveraging the Vision API for this project's custom use case proved to be a difficult task. By default, the Vision API returns a wide range of results, such as object annotations, image blocks, and extracted text. The main issue was that not every use of the API is identical. Some instances can return additional information, and some may return less data due to natural variation in image quality, image angles, brightness, and other unaccountable factors. Above all, the time constraint was another factor that proved to inhibit further production and enhancement in our project.

Accomplishments that we're proud of

Regardless of the handful of challenges we experienced, we managed to create a fully functional proof-of-concept and connective environment that would successfully extract the text from a license plate by known state-wide constraints in length. Additionally, logic was implemented to identify the region of the largest size (i.e., the license plate number) as it contains the largest font size on the license plate object. Finally, a database instance was established with data and was used to populate a user interface that resembles a real-world utility that can be used by service advisors at AutoNation.

What we learned

While we all come from highly technical backgrounds with differing levels of expertise and domains of knowledge, integrating our approaches yielded an overall successful product. Also, this project allowed us to work with several new and unused software and technologies (SceneBuilder, JavaFX, Blender, Google AI Vision, and Cloud SQL).

What's next for pyVision

This project has the potential to further expand on the existing cloud infrastructure to intelligently automate status updates, appointments, document signing, customer checkout, car inspections, any ratings, or reviews of service. Additionally, given the current situation regarding the COVID-19 pandemic, a health aspect can be incorporated to notify customers of high-density hours, reminders to social distance, locations of sanitizing stations across the AutoNation facility, and the ability to expedite/delay appointments in intervals (i.e., 15 min, 30 min, 1 hour) in response to high or low activity at a given location.

Share this project: