-
-
-
Lipnet demo
Inspiration
The driving force behind Airput was the desire to narrow the gap between differently-abled individuals and the rest of society. We recognized that many faced economic challenges due to their disabilities, and we aimed to empower them with a tool that could enhance their accessibility to technology.
What it does
Airput leverages advanced computer vision and LipNet to enable individuals with limited mobility to interact with personal computers using facial, eye and lip movements. This groundbreaking technology opens up new avenues for them to access and navigate digital resources with unprecedented ease.
How we built it
We embarked on this project with a keen focus on optimizing latency and model speed, ensuring that the tool operates in near real-time. This involved meticulous fine-tuning and optimization efforts, which resulted in a seamless user experience.
To develop our custom model, we initially faced a challenge with the dataset. It was primarily based on a single British individual from the Grid Corpus, limiting its applicability. Recognizing this limitation, we planned a creative solution. We would be extracting data from a diverse range of movies and TV shows, the we would be able to create organic datasets with pre-labeled information, significantly enhancing the versatility of our tool.
Challenges we ran into
One of the significant challenges we encountered was the limited dataset available from the Grid Corpus, which predominantly featured a single British speaker. This posed a constraint on the applicability of our tool. However, this obstacle prompted us to devise an innovative approach to generate new datasets from movies and TV shows, providing a broader and more representative foundation for our project.
What we learned
Through this journey, we gained invaluable insights into the intricacies of optimizing latency and model speed. This hands-on experience has deepened our understanding of real-time applications and the nuances of deploying advanced computer vision techniques.
What's next for Airput
Looking ahead, we envision expanding the capabilities of Airput by further refining our custom model and exploring opportunities for hardware integration. Additionally, we plan to transition Airput into a full-fledged open-source project, inviting contributions from the broader tech community. This step will ensure that Airput continues to evolve and make a meaningful impact on accessibility and inclusivity.
Log in or sign up for Devpost to join the conversation.