openSesame

The login page to the website
Taking a picture to compare to database
Raspberry Pi and Arduino which controlled the servo (to open/close the lock) and display

OpenSesame is a facial-recognition smart lock created by Timothy Yuan, Lucas Xia, Navid Mir, Toby Fischer, and Jacob Zhang for SB Hacks V (January 11, 2019 - January 13, 2019).

Inspiration

This idea was thought of to find a solution to students getting locked out of their residences or apartments, or allowing other people to retrieve items without the owner of the property having to be there.

What it does

OpenSesame is a security device that allows users to permit home access to other people efficiently. Using our website, users simply upload a single image of each person who can be admitted entry. Then getting access is as simple as letting the door lock mechanism take a picture of one's face.

How we built it

With Python Libraries OpenCV and dlib, we used machine learning to recognize faces and process them to simple data files. We built the server with Javascript, the database with mySQL, and hosted the server on Google Cloud Platform. The front end of the website was built with angular, while the door lock hardware used Arduino.

Challenges we ran into

Connecting all our separate projects together into one fully functioning system was the hardest step in our project. It required a lot of trial-and-error and quick problem solving. For example, we had to deal with faulty cameras, so we used a computer webcam instead.

Accomplishments that we're proud of

We used our diverse skill sets efficiently, splitting up and working on different parts of the project. This allowed us each to contribute to the project, which could not have been done without amazing teamwork.

What we learned

We learned how to use and integrate many of the most powerful and popular technologies of today, including Google Cloud Platform, angular, node.js, Opencv and other python machine learning/image processing libraries, and Arduino. While each of us came in knowing something, none of us knew everything and we learned a lot from each other's specialties.

What's next for Open Sesame

The facial recognition could be improved, in its accuracy and its inability to take in more than one face at once.

Built With

Submitted to

SB Hacks V
- Winner Arthrex - Best Security Hack

Created by

I worked on the initial opencv scripts and tried to get them running on our raspberry pi. It didn't have nearly enough computational power so I managed our instance of the Google Could Platform compute engine to host our website and run the heavy computing. Also set up a Jenkins server to automate deployment.

Toby Fischer
By learning Python opencv and dlib libraries, I worked on the face recognition scripts, specifically the scripts that take a user's picture, upload it to the server, and matches the image with the pictures in the database. Being part of this team was something I'm proud of; this project could have not gone far without the constant communication and collaboration of all team members.

lucasxia01
My role was primarily with the hardware and I also assisted with the server to hardware interfacing. I designed and soldered the various circuits needed for the project. I helped to write Javascript code on the Raspberry Pi to access the server response and to interpret this response as a digital signal that would then be sent to the Arduino. The Arduino would then use the signal to determine whether the lock, which would be turned using a servo motor, should be opened or kept closed. I also wrote Arduino code to display a messages on an LCD and to turn on signal LEDs, displaying whether the user was granted access or not.

Navid Mir
I worked on the face recognition/detection using Python and dlib/opencv. I also learned angular and managed to contribute a bit to frontend (styling with css mainly) before the deadline. I learned a lot; before my teammates showed me, I didn't know it was possible to do face recognition in just a few lines of Python.

jacob14916
timothyuan