Inspiration
I saw your contest, and I really need money and the first thing that came to mind when I thought about Meta VR glasses, "Do they have sign-language yet?"
What it does
This app can read your hand for the alphabet, and the phrase "I love you" and it can speak it out loud for people to hear what you sign. There is also a teach mode that can teach you how to do the alphabet. It also can hear English and transcribe it for the user.
How we built it
I built it using Claude, but i also designed how to train the reader, and what program to use. Claude wanted to use unity, but the Meta Spatial SDK seemed like a better fit. I wrote how it should judge each letter myself after giving and recording examples. It still needed some better instructions on how to see what letter the hand is making. I would prompt and then hand test myself to make sure it works.
Challenges we ran into
Some of the letters are similar to others, and it took extra work to really define the difference well. I use to install flooring for about 15 years and my hands are older and stiff, and they are not limber, so my hand actually hurts from trying to sign so much. I also, couldn't afford a deaf person to help me so the work I did is most likely wrong, but I have no way of knowing.
Accomplishments that we're proud of
Seperating T, M, N, A, and S was really difficult, but we did it with a very high success rate. I still had problems with R and U, but i figured it is just a proto type and I would want to train it on the actually vr glasses for it to make sense.
What we learned
I learned that I'm not good at sign language and if you hired me to finish the build or paid my company and gave me a budget, I would definitely hire a full tim deaf person to work side by side with me calibrating every word.
What's next for Nayerly: Offline Sign Language
I hope you hire me as an employee to finish this project. I have an IOS translator "Nayerly:Offline Translator" that I'm really proud of because it is full-duplex and I was able to make it lightning fast and now it is the fastest translator in the world by at least 3 times. I would add more languages to Nayerly:Offline Sign Language and the ability for it to translate inward and outward, and more sign languages would be added. I definitely think the transcriber is a very big weak point, so I would also like to upgrade that. Will you hire me?
