Inspiration
The inspiration is Jarvis, the home AI from Iron Man. Jarvis is interactive and doesn't just take commands so we wanted to make an AI system who used face and voice recognition and one that could just chat.
What it does
It uses the Kinect to detect faces in a color camera frame. It then takes a photo and runs the detected faces against a database of photos with associated names and gives names to the known faces (Project Oxford API). A message is sent over Microsoft's Azure web service to another computer which gets approval of the person inside and sends the response back. If approved, a raspberry pi receives a signal and turns the lock, unlocking the door and letting the person in. Other features include commanding Spotify through voice and having a chat with the Clever Bot api.
How I built it
It was built using the Kinect, Azure, Python, C#, Raspberry Pi, Adafruit Motor Hat API, Stepper motor, Spotify API, and Clever Bot API.
Challenges I ran into
We originally wanted to send signals over the locally hosted network, but since we're on campus we didn't have access to use open ports. Figuring out how to send signals over Azure's Service Bus Relay caused a big setback.
Accomplishments that I'm proud of
We finished the face recognition and identification, and got it to send a signal over azure! Unlocking the door was what we wanted to do and we accomplished it.
What I learned
We learned a ton about Azures IOT services and how to integrate tons of APIs together to make a truly unique project.
What's next for Project Navi
We're going to refine what we have and add tons of more features at our next hackathon. "Turn on the lights" is a a big one on our list. We also want to open it up for other developers to add onto, making it grow and become as life-like as possible.
Log in or sign up for Devpost to join the conversation.