Capsistant

What It Does

Capsistant is a smart hat that describes what you're seeing, powered by the Raspberry Pi. It uses the Google Assistant to deliver a personalized, dynamic audio experience based on contextual visual information from your surroundings.

Capsistant was submitted to the Google NYC IoT Hackathon 2018.

Built With

google-assistant-sdk
google-cloud-vision-api
raspberry-pi

Created by

I primarily worked on our applications of both the Google Assistant SDK and the Google Cloud Vision API, as well the interactions between the two. This involved the general sequence of events such as triggering the Assistant, taking over the conversation from the the Assistant if it is something that we want to handle, sending the to picture taken by our Raspberry Pi's camera to the Cloud Vision API, and using the Assistant to tell our custom replies to the user.

Stephen James
Nishir S
Thu Nguyen
Cognitive Science and Computer Science at UC Berkeley
Charlotte Chung
David Lie-Tjauw