RU You?

Inspiration

It's spooky season and everyone is terrified that the neighbor's peky kids will steal all their candy! With RU You? however, we can protect our kit kats, twix', skittles and M&M's from little Joshie's prying hands. Our invention helps you dispense candy only once per cheeky Annie.

On a serious note, packages being stolen from doorsteps is a very common occurrence and we wanted to solve this problem in a cheap, efficient and easy-to-use way. This inspired our creation, RU You?.

What it does

RU You? uses the Qualcomm Dragonboard 410c to take an image of the person trying to gain access. At the same time, we use Google Cloud's Google Storage API to maintain a database of faces that is used to determine whether a person is allowed to access it or not. This classification is carried out by multiple image analysis API's such as: the Google Vision API, the Clarifai API and OpenCV. The current image is compared with those in the database and the resultant score determines the course of action to be taken by the Dragonboard.

How we built it

Our approach consisted of two main components: Software and Hardware.

Hardware

The hardware workflow for RU You? can be summarized by the following:

Dragonboard touch sensor ==> Webcam initiated ==> Base64 encoding ==> POST request ==> POST response Analysis ==> Control Servo

Software

The software workflow for RU You? can be summarized by the following:

Image Processing on Dragonboard ==> POST request to Flask server ==> Scale image using OpenCV ==> Google Cloud Vision face detection and cropping ==> Query Google Cloud Storage database ==> Compare similarity using Clarifai API ==> Return response to Dragonboard and update Google Storage database ==> Control Servo

Challenges we ran into

We faced an onslaught of challenges with both the software as well as the hardware components of this project.

To begin with, the first Dragonboard that we used had a faulty network adapter and sapped away at 3+ hours of valuable time.
The connectors to the Dragonboard differed from the standard Arduino connectors and had to be specifically requested.
The libraries installed onto the Dragonboard to take pictures produced corrupted images. This was solved by turning saturation down to produce a greyscale image and cropping out the upper glitchy portion.
Retrieving stored data from Google Cloud Storage had us experiment with various features of the Google Cloud Platform's beta features.
Google Cloud Functions involved a very complex and involved authorization process when it came to hosting code and retrieving URL's from the Google Cloud Storage database. Our eventual workaround involved creating a Flask server and hosting it using ngrok/localtunnel as an endpoint. All communication between the Dragonboard and Google Cloud was done with this server as the middleman.
We had to figure out a way to communicate the images from the webcam to the GCP. Ultimately, the images were communicated using base64 encoding and POST request.

Accomplishments that we're proud of

We are extremely proud that we were able to integrate so many API's together! The hardware-software communication was definitely a challenge and we think that our solution was innovative and efficient.

One of our biggest accomplishments is definitely mastering such a large tech stack in such a short amount of time, especially everything we worked with was a novel concept for each of us. Not only were we able effectively make use of these individual components but also create communication channels between them.

What we learned

What didn't we learn? Almost every API, hardware component, analysis that we used we had to learn during HackRU Fall '18!

We also learnt that alot of people are mad about having their candy stolen on Halloween Night!

What's next for RU You?

We hope to extend this prototype to more applications and take advantage of it's potential for social good. This project can definitely be extended to security applications, storage applications, etc. We see this project reaching new heights in the near future!