EmoJibo - The Emotional Support Robot

Inspiration

When we first saw Jibo we immediately saw facial and emotional recognition as flagstones for the bot. These are meant to make him more personal and allow him to interface modern technologies like Postmates and Spotify easily and usefully.

What it does

Our Jibo searches for faces, detects emotion on those faces, then flags certain branches based on those emotions. For example, right now if Jibo detects that youre happy when youre around him, he will ask you if you want to throw a party- then text all your friends to tell them about it.

How we built it

We built it on top of a ton of APIs from RapidAPI and Microsoft. Using Node.js we trigger AWS Lambda functions to interface the different APIs and run them. Using Lambda we can change code from the ever changing API world on the fly without having to update on your robot friend. We utilized the different emotions to trigger different behavior trees that we think will make you feel better when you're down but great when you're happy. We have the emotional system built into the robot and we used animations to bring your companion to life.

Challenges we ran into

The first challenge for us was how inexperienced we all are with Javascript. We had a lot of problems getting Jibo to call Lambda functions and from there transfer data from the APIs that they call back into Jibo. We had problems training Jibo to look in the direction of a person and all related person detection functions. But most importantly, we are a team that formed out of groups of two. We had no idea how the group dynamic was going to work when we created this team and we are really proud of our teamwork and the experience that we all gained.

Accomplishments that we're proud of

First, we figured Javascript out. Second, like I said earlier, for a team that wasn't formed before MLH Prime, we work great together. I know that we will keep in touch in the future. Third, I think that the sheer amount of modern tech that we were able to get Jibo to work with is pretty cool. From Spotify to Postmates to Twillio and more, we are using the best tech made by the brightest people.

What we learned

We learned so much. From raw technical skills in Javascript and Emotional Image Recognition to softer skills like communicating as a team with people that you may not know. We know that what we learned here is going to be

What's next for EmoJibo - The Emotional Support Robot

We want to add functionality where EmoJibo gets to know you. We want it to be able to know what kind of music will cheer you up and what makes you less anxious. We think that the best thing in the future for EmoJibo is allowing other people to create tracks based on emotions. We want other programmers to be able to use our code to see if you're anxious then from there prescribe a solution.

Built With

• atom
• javascript
• jibo
• node.js