Kita-Kita

Inspiration:

The Idea of Kita-Kita ("Nakikita Kita", "I see you.") is our project to equalize societal norms, where normally deaf and mute people are forced to adjust (learn to lip read, etc.) instead of able people learning sign.

What it does:

Kita-Kita works by using computer vision to read and recognize FSL and ASL real time and translate it into Text and Text-to-Speech.

How we built it:

We built Kita-Kita using Flutter for the mobile app, MediaPipe for real-time hand landmark detection, and a custom TFLite model trained on FSL data for sign classification. Gemini API handles natural language assembly turning detected signs into full sentences and converting speech back into FSL sign sequences.

Challenges we ran into:

We had a hard time finding open source and available FSL (Filipino Sign Language) data sets which made it harder for us to have more accurate readings.

Accomplishments that we're proud of:

All our mentors were in awe and really appreciated the idea we presented to them. One of which had personal experience learning Level 1 FSL and really loved the idea of technology assisting people in need and equipping them with tools that they would use to equalize and communicate with other people.

What we learned:

We had an idea of having a learn page for non-deaf users to learn more about FSL and ASL to communicate better with deaf and mute Filipinos but along the way, one mentor told us that non-deaf people can't teach sign language. So our pivot was to keep the learning page very basic and very minimum, and have sign language schools as referrals in there, so they can officially enroll in ASL/FSL Classes.

What's next for Kita-Kita

What we plan on, if given the opportunity is to onboard and hire Filipino Sign Language practitioners who we can base the hand signs on and build a bigger library for FSL to be published open source for everybody to use freely, at the same time, ensuring employment and equal opportunities for Deaf and Mute Filipinos.