We live in a world with full of information and we don't always have the time to spare viewing a full video of who is visiting our front door like other door monitors provide. Instead Knocknock views the video for you and provides the information and context of the visit and sends the data to your smart device to notify you.
What it does
Knocknock provides users information about who is visiting their home by using OpenCV in conjunction with IBM Watson to gather relevant images and metadata to describe the visit. This data is then stored in a Firebase under the users account details where it is then pulled to smart devices to notify the users of the subject and context of the visit. Currently an app has been created for Android, IOS, Web and Fitbit to notify users. Knocknock also provides an in depth dashboard that provides users with details about all of their visits and additionally allows them manipulate their data using JupyterLab with the Knocknock JupyterLab plugin if they wish to analyze it themselves.
How We built it
Using an external camera, we take a video feed and use IBM Watson's facial recognition capabilities to assess whether or not there are people in front of your door. Then, using OpenCV and python, we process this information and make it so that it can be displayed comfortably on the display of a Fitbit Ionic. This information is then sent and stored securely in Firebase; here it can be easily accessed by our Fitbit application and Ionic apps.
Challenges I ran into
Making a JupyterLab extension one of the most difficult things I have ever done at a hackathon due to it being so new and lack of documentation. To learn how to build the extension I scoured Github for other JupyterLab extensions that used similar functions and did my best to learn from them how it fits in with the Knocknock extension. I also contacted some of the development team in the Gitter to learn from the experts.
Accomplishments that I'm proud of
I came into this hackathon wanting to learn how to build a JupyterLab extension and I am going to leave knowing how. It was a super challenging process, but I plan on building many JupyterLab extensions in the future.
What I learned
Sometimes documentation, guides and tutorials are not found on a silver platter. Instead you must painstakingly study others code and learn from them.
What's next for Knocknock
More metadata about the context of the visit, recognition of faces, integration with security systems, alerts about suspicious actions.