Created by Chang Liu and Ricardo Ferreira da Silva
This idea took shape after having seen the workshop for Google Cloud where the Vision API was previewed. Time and time again, people run into trouble with trying to capture a good group selfie where someone in the frame is not ready. We set out to solve that problem.
What it does
This app uses the built-in front-facing camera to take selfies and back camera for group photos. By using the Google Vision API, this app knows when to take photos by checking to see if everyone in the frame is happy.
How we built it
The app consists of two components: the android app and the web service hosted on Google App Engine. A photo is taken using the Android app from an Android phone, and then data is sent to the Google Cloud API to detect the optimal point where everyone in the group photo is smiling.
The Github repo contains two branches. The master branch contains the android app (which is currently malfunctioning). The web_backend branch contains the portion of the app running on Google App Engine. It also contains a test script using OpenCV to make use of a computer webcam.
Challenges I ran into
There were issues with deployment to the app engine as well as properly formatting the image data to send to the Google Vision API. Additionally, we had trouble with Android Camera API crashing!
Accomplishments that I'm proud of
We're proud of having working detection and having learned so much during the hackathon!
What I learned
We learned a lot about using Flask, the Android Camera API, and gained a deeper understanding of Google Cloud APIs (in particular Google Vision).
What's next for Smile!
We see two main improvements for this app:
- A feature where the app suggests an optimal angle to take the photo. This can make use of position and head rotation data that is also returned from the Vision API.
- Instead of sending the whole image to the Vision API, local facial detection could be used to crop just the faces and send them to the API. This would cut down on data usage and processing time on the cloud.