-
-
Celebrity recognition of the famous Ellen Oscar selfie. (Made using Amazon Rekognition)
-
Celebrities in the Office cast. (Made using Amazon Rekognition)
-
Web Interface
Inspiration
We were inspired by music detection apps and widgets like Shazam. We wanted a similar application to help identify movies in situations where even your film buff friends can't figure it out.
What it does
Our internet of things (IoT) device takes video straight from the Raspberry Pi Camera and uses Amazon's Rekognition platform to identify celebrities. It uses this library of celebrities to search IMDB and find a movie that they all performed in.
How we built it
We used AWS S3 and Rekognition to power our computer vision and video analysis. We took advantage of Rekognition's native celebrity recognizing methods to pick celebrities out of videos and images. We then cross referenced the detected actor's IMDB profiles to find movies they had appeared in together and determined what movie was in the uploaded video. Our development name was Krakatoa, a play on Shazam.
Challenges we ran into
Amazon Rekognition's celebrity face library is limited and would often result in a false positive as another celebrity or just not be recognized all together. We solved this by filtering results by confidence scores provided by Rekognition and reducing the number of actors when cross referencing movies. This allowed us to look at different combinations of only some of the actors in the scene instead of all of them at once. We found this produced more accurate results. We also ran into issues live streaming the Pi Camera feed to our web portal. We had to use some creative solutions and call upon the help of a mentor. We ended up using traceback's in order to properly debug while using the flask library.
Accomplishments that we're proud of
Successfully implementing our IoT device using cloud computing and computer vision in AWS. Seeing it give a correct identification of a movie is extremely fulfilling. We have great expectations that this could be a standalone device or app with the cultural significance and popular usage of Shazam.
What we learned
We learned that there truly is no debugging tool to rival logging. Being able to see where your code is getting stuck is so helpful. We also learned more than expected about internet connected devices and applications. Making all our platforms, devices, and languages mesh smoothly and in a more reliable way took hours of fine tuning.
What's next for Quik Flik
We would like to port our application to both iOS and Android in order to expand our user base. We would also like to implement the Python solution on a Raspberry Pi Zero W for a more compact and clean looking product. We also need to optimize our algorithm for speed.
Log in or sign up for Devpost to join the conversation.