Real-Time Object Identification using Machine Learning

YOLO Object Detection example

Inspiration

We were inspired to do this project upon discovering FaceAPI, a simple javascript facial recognition implementation. Upon exploring the FaceAPI, we began to test object detection and eventually landed on DarkFlow, an implementation of the Darknet pretrained neural network. We wanted to create a multifaceted application that can read not only faces with the FaceAPI, but further implement Object Detection to create a powerful robust application suited for many needs, with a future potential for more.

What it does

With the FaceAPI and Darkflow network not yet implemented simultaneously, the FaceAPI simply scans a face using the webcam to be able to accurately determine the mood a person is in, ranging from happy, sad, angry, neutral, to surprised. The Darkflow Object Detection network also uses the webcam to register the environment and identify items in view (people, objects, etc.).

How we built it

The FaceAPI was a simple yet powerful javascript program that was straightforward in implementation, utilizing the webcam to draw models for faces and compare to the model requisites for moods to try to accurately identify the mood a person was in. The Darkflow object detection application was built using Darknet, an open source neural network framework written in C and CUDA, implemented using the Tensorflow API and YOLO, a neural network which does the identifying of objects.

Challenges we ran into

The biggest issues stemmed from the dependencies Darkflow required to be able to run, which included older versions of Python and Tensorflow, and it required multiple modules to be installed to run correctly, which required for older versions of those modules to be implemented as well. Lack of proper documentation only made this process harder, and required for us to search hard to be able to resolve any compatibility issues. Our first attempt also had us stuck at trying to get our own dataset into Darkflow to be able to identify objects we wanted and not what the network had first pre-trained, which seemed to be the incorrect approach and multiple errors made it impossible to pursue anyway. A different approach all-together was required, but the time spent on the first approach led to little time left for other things we wished to accomplish, such as Data Analytics and GPU implementation. The final challenge to describe might just be how slow the frames are on the current CPU version of the Darkflow application, which again would be fixed through GPU implementation.

Accomplishments that we are proud of

Getting the entire neural network to work was quite the accomplishment, as at first we could not get it to distinguish anything in the environment, and it was only able to default and identify one thing. The FaceAPI works very smoothly as well, and was the basis for the project overall and what we ended up creating. This application has potential to do much more, which we are definitely proud of, and hope to accomplish in the future.

What we learned

We learned and had our first legitimate experience with Machine learning, and learned to appreciate the very simple applications we use today that implement the concept considering how difficult it is to program and train. We were able to learn multiple APIs and Tensorflow itself as a powerful API that uses datasets like COCO and models like RCNN and YOLO to create powerful object recognition applications, and we also learned how to communicate effectively and do our best to identify and resolve issues we faced.

What's next for Real-Time Object Identification using Machine Learning

As aforementioned, this application has so much more potential, and we hope to be able to implement data analytics into the application in the future (record moods and store them for research for customer reactions to products, and more), we intend to combine the functionality of the FaceAPI and the Object detection, so it is possible to use within one application, we hope to one day add image processing as well as a quirky yet appreciable feature that is also quite similar to Face and Object detection, and overall hope to speed up the application as well as train it further, so it is smooth and more usable.