Inspiration

Often in our lives, we are surrounded by boring ads. This is a sad fact as tremendous amount of money is being wasted for ineffective advertising. For this reason, we decided to create Spegill - an interactive web application capable to remembering aspects of user's life by having simple verbal conversation and advanced facial recognition algorithms. Using Indico, Wit.ai and Face++'s API along with Flask and node.js, we aspired and have created an intuitive AI with machine learning.

What it does

Spegill is capable to determining a user's characteristics and preferences using machine-learning. As well, it is capable of remembering each user based on facial recognition without the need to register or authenticate. The web app is able to determine and record all sorts of data about the user's preference without the need of typing anything at all. This enables Spegill to show relevant advertisement on the fly as user walks by a Spegill enabled banner.

How We built it

We used node.js and Flask as our major frameworks. We also reversed-engineer Wit.ai's Javascript SDK to receive raw information we later incorporated into our app. We used APIs from Indico, Face++ and Wit.ai.

Challenges We ran into

We attempted to work with Azure Machine Learning's Face API at first but we failed to get it to work in the end. We also ran into many issues trying to get the facial recognition to work the second time when users come back.

Accomplishments that We're proud of

We created an AI that is very similar to those of many large companies in a effective manner. We were also thrilled about having the AI being capable to recognizing people multiple times and having input directly through voice.

What We learned

We learned more about the various API we used as well as Azure Machine Learning's Face API. Personally, I finally had the chance to work with Flask and Python with the help of my team members.

What's next for Spegill

We plan to integrate a full advertisement system that will effectively choose interesting ads for each user. We hope to make the interface completely hands free in the future as well with more fluid interactions.

Share this project:
×

Updates