SafeHouse AI

Inspiration

One of our teammates,is constantly worried about leaving his doors unlocked when leaving or after entering his house. We created SafeHouse to alleviate his concern and those of others like him. The data created by SafeHouse can also be used by insurance companies to better calculate risk and assess damages, which will make SafeHouse accessible to all users for free.

What it does

SafeHouse AI leverages the security cameras already around his house and gives him the power to check the state of his doors through his Google Assistant and Google Home. When we realized that door locks look similar to stove knobs and garage locks, we decided to push it further and have those covered by SafeHouse as well.

How we built it

We collected a large dataset of door lock images in different states, and used a pre-trained ResNet34 model to fine-tune a Convolutional Neural Network in a Transfer Learning scheme to create a model that classifies their various states. We then deployed the model the PyTorch. The front-end is the Google Assistant/Google Home and we programmed our key commands through DialogFlow. Upon demand, the Assistant fires a request to our cameras, which takes pictures, crop the locks and sends it to our classifier. The classifier serves the pictures back to the Assistant along with a prediction of the lock states.

Challenges we ran into

It was extremely hard for us to figure out the cloud services(AWS, GCP) when deploying our classifier. And we ended up putting our classifier on Docker and then hosting the Docker on GCP.
Since the Google Assistant was our only user interface, we also had a lot of trouble dictating its behaviour to make it intuitive to the user.

Accomplishments that we are proud of

It was our first time doing something of this complexity in such a short time. SafeHouse AI uses IoT, ML, and the cloud, topics our team were not particularly familiar with. We're proud of how polished our end product is, it is already in a very usable state.

What I learned

As a team, we learnt a lot about how IoT can provide a lot of conveniences and possibilities for the users. We also learnt a lot about Machine Learning and working with Docker and the various cloud platforms, all team members an expertise in a specific topic, and it resulted in massive exchange of knowledge.

What's next for SafeHouse AI

We want to continue to add features to our product, and transform into a comprehensive IoT home security software. In the near future, we want to add geolocation using IFTT to immediately notify the user when they have left their home in a dangerous state(unlocked doors, stove open, and other near future features).