Inspiration
I (Naman) love to write screenplays. I do this very often in my free time and one day as I was writing a screenplay for a crime drama I remembered a really good story that I had told a friend many months ago. But I forgot most of the details. I wish I had recorded all of my conversations so that I could find out what I had said in the the past, but I realized that even if I had it would require a ton of time to search for what I needed.
So I called up my friend (Shaurya) and told him that once our classes ended we would solve this problem for a Hackathon.
What it does
Our app accepts a video recording of your day, stores it into the cloud, uses voice recognition to detect key works like "idea" and laughter, and uses image detection to detect clips where the observer smiles. Then by filtering these clips, our app will determine the timestamps of the best moments and create a compilation of the best clips of the day. Then we will send you an SMS containing the link to the video so you can watch it directly on your phone!
How we built it
We used Android Studio for creating a working Android app, and AWS to host our backend Machine Learning API's and for storing our videos. Since our goal was to store 24hr long clips with an AR headset, we needed a way to increase scaleability. For the Machine Learning we used Amazon Rekognition for detecting smiling faces in the input videos. We wrote a python script using speech_recognition API. This allowed us to detect keywords and choose timestamps in the video accordingly. Using those 2 systems we simply wrote another python script that helped clip highlights from the the large video uploaded by the user. All of our python scripts were run on a Docker image that was accessed by a Lambda function in AWS. This allowed us to create Topics that when completed, we could immediately send an SMS message to a user. Then the user could simply watch their highlighted clips of the day via links from the S3 public bucket! YAY!
Challenges we ran into
We wanted our entire app to run on an AR headset so that our users don't need to hold up their phones to record their day. The app would be automated and the AR headset would record on its own. We planned to purchase a Microsoft HoloLens headset as students but were unable to place an order. Therefore, we decided to create an Android app instead, a consequence being that we needed to hold up our phone during testing.
We wanted to send messages to the app directly using AWS push notifications but were unable to find solutions that worked. We decided to use SMS instead.
Since we had only 2 days to submit we could not do an entire day of video recording. For our demo we only recored for 30 mins.
Accomplishments that we're proud of
We have now mastered AWS Lambda functions and S3 storage. We are confident enough to create apps for a real life business model and are proud of our journey here.
What we learned
The bigger the problem the better the solution: We found a solution to a problem that would bring joy to a creative persons life and our own. A simple solution is not simple: We thought we could put together some brand new API's and cook together a brilliant product. We spent a ton of time reading articles since we were dealing with new technologies and had to reject a ton of optimal solutions because some of them were very complex and we didn't have a ton of time to learn all of them. Given more time, we could have created a more refined project.
What's next for Remember This?
The Skys the limit! We NEED to get this on an AR system so that we can essentially have a handsfree program. We need to then find solutions to send messages to our app, and polish up our application performances. The most important thing to do is find ways to compress video so that it doesn't occupy so much precious space. We can market this product for different groups of people by creating optional parameters each user can select such as happiness, sadness, or excitement. Right now we are forcing users to assume that when their subjects smile, laugh, or say the word "idea" they are experiencing an important moment in their life. But for a playwright, sadness might be something more important to record.
Log in or sign up for Devpost to join the conversation.