CelebDetect

Machine learning based web application to find all instances of given celebrities in a video!

Comment 1

Inspiration

Our team wanted to get experience in making an innovative and practical full stack application that utilizes machine learning, deep learning neural networks, multiple APIs, and a cutting-edge web framework. We did exactly that with CelebDetect.

What it does

CelebDetect allows users to upload video files and find all instances of given celebrities. Users are prompted to upload a video file and then search for one or many names of celebrities using just their voice with help from the Google Cloud Speech-to-Text API. CelebDetect then utilizes uses Amazon Rekognition to scan all faces in the video and returns all the points in which those celebrities are in the video. Users can jump around between points in the video in which their given celebrity or celebrities are in the frame!

How we built it

The front end was built using Python Flask, JavaScript, HTML/CSS.

The back end utilizes the machine learning based Amazon Rekognition API, tuned to detect as many celebrities as possible in different settings, cosmetic makeup, and other conditions. The video file that is uploaded is stored using Amazon Web Services and LeanCloud.

Users search for the name of a celebrity or celebrities using the machine learning and neural net powered Google Speech-to-Text API.

Challenges we ran into

Since we are using many tools, connecting all these different components was an omnipresent challenge in which all of our team members had to "wear different hats".

Accomplishments that we're proud of

Our team is very proud of making a useful and innovative product that connects many languages and APIs. The front end is aesthetically pleasing and runs on modern technology such as Flask, JavaScript, and HTLM5. The back end utilizes two advanced APIs from Google and Amazon that implement cutting edge technologies like machine learning, natural language processing, and automatic speech recognition (ASR) powered by deep learning neural networking. These two ends communicate quickly and efficiently by taking advantage of the cloud, powered by Amazon Web Services.

What we learned

We learned how important constant communication is in building software that relies on almost ten different languages/APIs. We utilized the Agile software development process and extreme programming that allowed us to be efficient.

What's next for CelebDetect

Publish the web page on a publicly accessible domain for the world to enjoy!

Built With

Share this project:

Updates