There are always times that your friends decided to pay you a visit yet you’re not at home and both of you forget that happened afterwards. There might as well sometimes that someone is at the door but you’re simply in a mess that you can’t even look at who that is. You sometimes might also wonder who has been constantly visiting you, and if that “you” is a small business owner, you probably really want to know if the guy is a returning customer to your business. PandaWatcher is the solver to those problems. It will look at the faces of all the visitors you’ll get, extract and store the data of the visitors and eventually return them to you in a simple and neat website.
From the technical side, PandaWatcher is a computer vision program that recognizes the faces of visitors, store relevant information about them and return those information to the owner. On the hardware level, it only requires a first generation Kinect, a microphone and a Linux machine to operate. The software part of the program consists of 2 sections: a computer vision program that recognizes human faces and a simple website that returns information of visitors such as the name of the most frequent visitor, the average age and the gender composition of visitors to the owner.
We build the computer vision program based on Kinect and Cognitive Service library from Microsoft and OpenCV. To ensure the interface is more user-friendly, we implemented audio-to-text and text-to-audio features using methods under Speech Recognition and Cognitive Speech from Microsoft.
The website was built on HTML and JavaScript. To store as much information as possible, a database was implemented. The computer vision program will write data into this database when a visitor shows up. The website will draw data from the database whenever a request is made.
Although we were able to come up with a working program at the end of this Hackathon, there were many challenges that almost drove us desperate during this 36 hr marathon. There were libraries simply refused to use a microphone when it was directly called upon. There were also times that the two essential libraries we need exclusively support two different version of the language we use. We also had a very long struggle in determining the parameters we wanted for the facial recognition. Yet after all we were able to stay calm and conqueror enough of them to make the program up and running. Such spirit, along with the fact that we managed to complete a working prototype, is the accomplishment we are the most proud of.
However we all understand that PandaWatcher is not perfect. Database is an entirely new concept for almost all of us on our team. Both the website and the computer vision program is done in a haste that requires much optimization. If more time is granted, the following is a list of actions we already had in mind for PandaWatcher: Optimize the handling of video so that the users may view the stream from the camera. Implement a better auto-correction algorithm for facial recognition. Currently we only consider a “face” is a new “face” if we receive two positive return in two continuous iterations of facial recognition. This algorithm, though can improve the accuracy to some degree, we still wish to increase the number of test cases to further that accuracy. Consider to move this application to platforms other than Linux. The first target platform would be Windows since we’re using so many Microsoft related libraries.
Log in or sign up for Devpost to join the conversation.