Automatic AI curated photos
Inspired by the concerns people have over technologies pushing people apart, we wanted to build something that brings people back together. We noticed people wasted hours struggling to get the perfect shot on parties and events while that time could have been spent on having fun with friends and meeting new people. It's even harder when it comes to group photo as there's always one person that needs to be holding the phone. We were inspired to create an AI camera using Microsoft Azure Face API that takes photos whenever it detects happiness and generates the best candid shots for you to share on Facebook and Instagram. This is especially useful for people with disabilities as placing the phone at a static place is all it takes, and CANDID does everything else for you.
What it does
CANDID is an AI camera app that uses Microsoft's machine learning to algorithmically find the most special moments in a social gathering. Simply turn the app on and it will automatically capture moments when subjects are experiencing spontaneous joy and happiness. Just select your favorite picture from the highlights captured by the AI and then instantly upload onto Facebook and Instagram. Redis Queue in order to manage background AI inference tasks.
How I built it
We built a cross platform mobile application using React Native to capture continuous streams of images. We send the images to Microsoft's Azure Computer Vision API in order to measure the emotions of all people in the image. We use a Python Flask server to calculate recommendations of the best photos and we used the Facebook API to login to the app and to share the photos on Facebook and Instagram.
Challenges I ran into
Due to ever-changing SDKs and versions of react-native, the Facebook React Native SDK was difficult to get started with due to novel problems introduced and conflicts between the source code of the SDK and React-Native. Recent versions also made debugging more difficult, since there wasn't a lot of information online to help. There was also some difficulty with HTTP requests through React-Native, with one of our developers never having worked with REST APIs before. Through collaboration and teamwork, as well as talking through our difficulties, we were able to work out any issues that arose.
Accomplishments that We're proud of
We're proud that we built a fully functional application with beautiful UI using cutting edge technology for both Android and iOS smartphones in just under 36 hours.
What we learned
Sleep isn't as necessary as we thought it was.
What's next for CANDID
We believe this app could be very useful at different public events. Installations at clubs, weddings and school events could make it so simple to take awesome candid photos. With automatic photo taking people can spend more time just enjoying their event, and also save money without the need to hire a professional photographer.
We would also like to develop the different modes available within our app. For example, we would like to develop a function that finds the right timing to take a group photo to avoid people blinking their eyes, a mode that zooms into interesting areas in the scene, and a mode that automatically adjusts exposure and color balance.
Lastly we want to implement an auto posting option that automatically uploads photos to Facebook and uses Facebook face recognition to tag people who are in the photo. This is efficient for event page where there are large numbers of photos being taken as it usually takes a long time for the event organizer to review and upload those photos.