1.For the last couple months, I've been thinking about the long term effects technology may be having on our brains. Do you feel that the widespread use of calculators has decreased the average humans ability to do mental math? Do you think the use of GPS has weakened individuals spatial memory and their navigational skills? I certainly do, and I believe it's largely because technology has been made to replace the work our brains for training. If we had some technology to enhance our brains ability to do mental math, or to give more of a reward for having gone through the work of not using a calculator, I think our brains would be stronger than ever.
I have a MASSIVE fear of memory loss. I hate the thought of forgetting or being forgotten by the ones I care about, so I wanted to find a way to understand how memory works in hopes of enchancing the brains ability to do math. The goal of this proof of concept project, is to bring attention to the benefits that gathering datasets combining visual info with recollections of memories could bring. I believe that by comparing multiple video segments taken at times of recollecting an individual's memory of an event in the past, we could identify some key tags/details that were memorable about the event. We might even gain a better understanding of what the brain is storing subconsciously by allowing machine learning to show us features in videos we may not have noticed before.
As the field of augmented reality grows, more and more people could be wearing cameras throughout the day, recording their point of view. From what I understand, Amazon's "Alexa" keeps a fixed segment of audio over time, in order to detect if someone is saying "hey Alexa" so it knows when to interact with the user. This gave me the inspiration to come up with the idea to record a fixed segment of video to be stored with "memory tags" identifying what event a user was reremembering.
My creativity tends to center itself around finding tricky ways around doing work thats too hard for me to understand, or stuff that takes too much time. Although we've made incredible progress in understanding the brain biologically, I wanted to find a way to understand the brain that lined up with my passions for psychology, augmented reality, and machine learning. I believe these passions were strong influences in the shaping of my idea.
What it does
I don't have a huge dataset of thousands of reremembrances with memory tags, so originally, I thought this idea wouldn't be feasible for a really long time. At the Google Cloud workshop, I realized that I could mimic the many video segments someone would collect during re-remembrances of a specific event, with the impressions of an individual event on the masses. My hypothesis is that hashtag's on twitter may help reveal what sticks in people's minds about certain events they experience.
2 Functionalities: Scrapes twitter for tweets, isolates video links, converts indirect video links to direct links(t.co/gibberish -> youtube.com/w...)
downloads scraped videos over the cloud, and funnels them to Google's video intelligence API, which classifies segments of the video with different labels.
I collected sets of labels from thousands of segments, collected from videos taken under the hashtag "#Coachella"
How I built it
Found some scraping libraries for social media apps, lots of trial and error and modifying before being able to successfully capture video links. No other libraries I found were able to isolate direct video links as mine was able to.
Researched Cloud API for video intelligence, practiced examples, modified some inner code files of the models before realizing there were much higher-level functions for what i was trying to do?
Challenges I ran into
alot of trouble pertaining to authentication with GCP, link conversion with Tweepy, and data collection from convoluted json files.
Accomplishments that I'm proud of
I was able to finish a sizeable project on my own, despite losing 3 team members prior to the competition due to midterms next week. I decided to do the competition despite having a midterm on Monday, and I'm proud I followed through with taking on the extra studying and hard work to be able to work on something I'm passionate about.
What I learned
There's so much more to learn from Cloud APIs and I plan on using everything I've learned in the future