Capsistant

What It Does

Capsistant is a smart hat that describes what you're seeing, powered by the Raspberry Pi. It uses the Google Assistant to deliver a personalized, dynamic audio experience based on contextual visual information from your surroundings.

Capsistant was submitted to the Google NYC IoT Hackathon 2018.

Built With

raspberry-pi
google-cloud-vision-api
google-assistant-sdk

Created by

I primarily worked on our applications of both the Google Assistant SDK and the Google Cloud Vision API, as well the interactions between the two. This involved the general sequence of events such as triggering the Assistant, taking over the conversation from the the Assistant if it is something that we want to handle, sending the to picture taken by our Raspberry Pi's camera to the Cloud Vision API, and using the Assistant to tell our custom replies to the user.

Stephen James
Nishir Shelat
USC Computer Science & Engineering Student
Thu Nguyen
Cognitive Science and Computer Science at UC Berkeley
Charlotte Chung
David Lie-Tjauw

Updates

Nishir Shelat started this project — Jul 21, 2018 01:19 PM EDT

Leave feedback in the comments!

Log in or sign up for Devpost to join the conversation.