We wanted to build an embedded system but with features that couldn't be accomplished with the onboard computing power. So we brainstormed different ideas that took advantage of Google Cloud Platform. Google Cloud Vision provided power that couldn't be accomplished with the Raspberry Pi alone. We realized that pet owners paid for overpriced products such as smart dog doors for little in the way of features. So we created a dog door that offers more features for less than a quarter of the price.
What it does
The Raspberry Pi uses the Pi-camera to take photos, which it then sends to Google Vision to process the images and send back relevant information about the pictures. The Pi then parses the information and determines whether or not there is a pet or person in the photo. If either of those is found, then the Pi stores the photo using Google Cloud Storage and generates a publicly accessible link to the image. Next, the Pi communicates with Twilio in order to send an SMS message with the image and some flavor text to the user. When a person is detected within the taken photo, the system outputs an audio file of balloons popping in order to alert UCI PD of an intruders whereabouts.
How we built it
Initially, we split into pairs with one pair focusing on setting up the Raspberry Pi while the others read through documentation in order to begin interacting with the Google Cloud Vision as soon as possible. Once the Pi was booted up with Raspbian, we each set to work on a different aspect of the project: communicating with Google Vision and parsing the JSON response, identifying whether the subject was a cat, dog, or person, and sending the user an appropriate text message through Twilio API. Once we had
Challenges we ran into
We met a wide variety of challenges from the get-go. None of us had experience with Raspberry Pi nor did we have a monitor. Therefore, we had to start the Pi in headless mode and ran into complications with booting the OS remotely with SSH. We reached out to mentors to help us troubleshoot at this time. While some of us worked on tackling this problem, the others faced early roadblocks by having to parse through documentation due to our lack of experience with Google Cloud Platform. Early on, much of the setup was also reliant on itself sequentially and thus tasks could be delegated and done concurrently. Once we were able to boot the OS and begin working on the software, tasks became much more efficient. From there, the bulk of the obstacles we faced stemmed from our lack of understanding of toolkits and APIs we were utilizing; these problems were resolved as we continued to learn and work with the platforms.
Accomplishments that we're proud of
This project posed various challenges for all of our group members. Trevor had struggles with implementing the Google Cloud Storage service and creating the bash script, but managed to do it in the end. William managed to learn how to use multiple API's and create the starting ground for most of our functions. Danny parsed the JSON files that were sent from Google Cloud Vision, set up the Raspberry Pi to handle headless mode and set the pins on it in order to control motors to lock the dog door. Finally, Daniel integrated the Twilio and Google Storage API to enable communication with the client. He also managed most of the Github merges.
What we learned
We learned how to integrate a variety of APIs such as Google Cloud, Google Vision, and Twilio into Python. We also gained experience on how to use Rasberry Pi in conjunction with the cloud. The whole project made us become more familiar with Python and language capabilities. For Twilio, we specifically learned to send the message and read particular messages to instruct the program accordingly. For Rasberry Pi, we tried to maximize its potential by utilizing its sound features, camera features, and its given environment of development (geany). For Google Cloud Storage, we tamed it so that we can make our data public, and able to used by different software. Integrating APIs pose their own challenges, but we are able to parse the data or work around with its given functionality. Beyond that, we learned how to work with bash script to automate the process of running program.
What's next for Pet Portal
Something important that we didn't have the capability of implementing during this Hackathon is using the Pi's GPIO pins to turn a motor in order to lock/unlock a door. In addition, we wanted to create a more personal experience for users through further interfacing. This could be implemented by means of text messaging or mobile/web application. Some features the user would be able to configure are specifying pet breeds to maximize the accuracy of their unique pet being detected, locking the Pet Portal remotely at the users' discretion. More efforts could be made towards streamlining the process of starting the Pet Portal application on the Pi's boot up.