Humbly beginning as creating a house with personality that could capitalise upon cutting-edge deep learning and computer vision and combining it with advances in understanding of psychology to create the definition of an interdisciplinary solution to a wide array of problems.

What it does

SmarterHome requires only a small amount of hardware to setup and get started. Using a dedicated camera (or network of cameras - with it's modular and expandable design) it will track in real-time the occupants of a room and upon recognising an occupant of the home will load up a dedicated profile of light settings unique to that individual.

How we built it

Using a convolutional neural network we take input from cameras and determine in real time the number of and names of (based on a set of registered users) everybody within frame. Perfectly expandable using Qualcomm Dragonboard 410c chips in order to provide a 360 degree guarantee of coverage.

Once finding an occupant that it recognises will poll to a NodeJS server running on a Qualcomm Dragonboard 410c in order to compare to the dedicated user profiles that are established via a web interface running on an external web hosting provider.

This in turn will return a final request to the NodeJS server with the configuration for lighting in order to instruct the board to switch outputs as appropriate.

Challenges we ran into

Networking with the Qualcomm board is temperamental and requires strong refinement - a subset of the team spent several hours deciphering and understanding the somewhat under-documented protocols and environment of the chipset.

Version control with git was more complicated than a traditional development environment in this project due to the extremely modular nature of the project with a distinctive lack of any integrated parts until the very end of development - meaning that as a team we could split up and work on individual projects with version control that was self-managed and simply share final, release-grade code to each other at the end.

Accomplishments that we're proud of

Taking a chipset which we had no prior experience with and developing a complicated and specific NodeJS solution with it was both intellectually challenging and practically complicated in terms of managing a wide variety of skills normally unrelated to each other - practical engineering skills with theoretical implementations of programming.

Implementing a 75 layer convolutional neural network feature over 3.7 million parameters to provide accurate, real-time facial recognition and developing it in record time. A simple implementation of a network of this scale has the potential to cause substantial problems and time delays, thankfully we avoided those pitfalls.

Combining environmental friendliness with increased accessibility with a more homely feeling for everybody when they come home. We reduce energy wastage by turning lights off when not in use, we allow for photosensitivity and other similar accessibility requirements to be automatically catered for in work and home and knowing that your house knows you allows for a comforting and relaxing welcoming when you walk in the door.

What we learned

A whole host of things about deep learning and software development in a team environment. From how interactions between servers take place successfully to the shortfalls of python's indentation requirements in nano - we cracked a lot of bugs.

What's next for SmarterHome

Dedicated accessibility enhancements to provide individuals who are photosensitive an automatic adjustment and comfort when they enter their home or business.

Increased modularity for the output from the Qualcomm chipset.

The ability to automatically identify, report and notify you or the police upon unaccompanied access by a stranger to you home.

Built With

Share this project: