Prototype of sensor board
Architecture of the system
What it does
Zenda provides a reliable platform for gathering and analyzing data gathered from clients connected to sensors. As safety is one of the key things when it comes to Smart House, it natively supports facial recognition logging in and verification through registered users' fingerprints. Gathered data can be easily modeled, as for example we implemented in-house temperature adjustment through a linear-regression machine learning algorithm. Our hardware part of the team has used the Raspberry Pi platform to build two sensor stations communicating with the main server and feeding it with the gathered data.
How we built it
The main server provides all communication with the remote sensor stations, facial recognition server, and machine learning server, as well as it delivers code for data gathering and modeling. It is build using flask, JQuery,HTML5 and CSS3, as well as AJAX. It communicates through JSON with the other instances of the system. Sensor servers are hosted on RaspberryPi platform, where all communication with the sensors is done using I2C and GPIO on parallel observer threads, packing data into JSON structure returned by the flask server on request from the main server. We have built python code for handling data from proximity sensors, air humidity, air quality, and temperature sensors. Computer Vision part is made using OpenCV and flask. The request for facial recognition is sent from the main server to the computer vision server, which checks log-in photo using algorithm trained on the dataset of photos of faces of verified users. Arduino handles fingerprint scanning and comparing it to the base of fingerprints of registered users and it sends the verification data to one of the sensor stations. Our machine learning part of the project is based on linear regression using pre-generated example data-set of temperature preferences of the users, to automatically select the temperature on the thermostat.
Challenges we ran into
We fried one of our raspberry pi's... :( But actually, one of the greatest challenges was to provide a reliable dataset of pictures of faces of registered users, in such a way our computer vision part of the project would successfully detect them. Communicating all parts of the system was also a very demanding task, especially on sensor stations, which had to handle multiple threads at once - both for reading data from sensors and communication through the Flask.
Accomplishments that we're proud of
We have working face recognition, to log in into dashboard! We have software support for cool sensors, like the fingerprint reader or air quality sensor. Communication between all various parts of the system works regardless of the hardware and software platforms.
What we learned
Patryk did Computer Vision for the first time in life. Maciej and Wojtek had no prior experience with python and flask and had so much fun with writing sensor station server code. It was all done thanks to great teamwork and it was the main skill we have all learned through this hackathon. Mutual support can help overcome even the biggest technical issues.
What's next for Zenda
-Support for more sensors -Creating more beautiful frontend -Gathering better datasets for machine learning subsystem