We aspire a world where AI uses speech, as well as visual and behavioral cues to interact with humans, thereby providing an emotional connection.

What it does

It is an artificial intelligence agent that watches your front door with a web cam, recognizes your face and emotional state when you come home, and responds with an appropriate greeting and musical selection. If it doesn't recognize your face, it calls your phone and sends a text with a photo and an intruder alert, and if it sees pizza it announces pizza delivery!

How we built it

Train an AI model with #Clarifai, and #AzureEmotionAPI to capture emotions. Leverage #Nodejs and #WebRTC for the image capture, and #Github, and #Heroku to host. #Twilio and #Slack to power the communications, and #Hack for the spirit!

Challenges we ran into

  1. Training set for custom #Clarifai model
  2. Capturing emotions in the general #Clarifai model
  3. Trying to create state of the art NN emotion model from scratch (using #Keras) to create an ensemble model
  4. Hosting #Twilio event handlers on #Heroku

Accomplishments that we're proud of

What we've accomplished over a weekend!

Share this project: