We, as a team, have always been interested in communication, more specifically, how can we better communication around the world. This is why we thought of NPCIRL, an app that allows users to have live subtitles for their video calls, youtube videos and even presentations.
What it does
NPCIRL is an app that provides live subtitles over any kind of video, even live ones, recognizing the people talking and putting the text over the appropriate person's heads. This means that if there is more than one person on the video, then it will put the correct subtitles over the correct person's head. This is a program that, with some fine tuning, could change the way we communicate around the world. It would also help audio-impaired individuals, allowing them to go anywhere and providing live subtitles at their location.
How I built it
We built it using python and its many libraries, with some such as the one that allows us to connect to google cloud to call their speech to text API, as well as others that allow us to use facial recognition. Our facial recognition software is also able to detect mouths, allowing for accurate subtitles for each person in the video feed.
Challenges I ran into
We ran into some significant challenges throughout the project, a major one of which as dealing with a disconnect between the installation of certain python libraries. This is because certain python libraries are simple to install on an Linux but extremely difficult on a windows computer.
Accomplishments that I'm proud of
We are very proud of the project as it allowed us to take a deep dive in to our area of interest, as well as learn many new things about python, its libraries, and google cloud. One major accomplishment I am proud of is our seamless integration of all of the very different parts of the project, allowing us to create a single cohesive program.
What I learned
I learned a lot about how facial recognizance software works and how it can be used for the betterment of society. Through our work, we also learned about many different Python libraries and how to use them, which will not only benefit us now, but anytime we use Python again in the future as well.
What's next for NPCIRL
We were thinking of adding the ability to translate on the fly, which we would be able to do through an API call to the google cloud.