Nowadays everything is done on the Internet: online banking, online shopping, online application, etc, which forces the middle-aged population to get online. While some adapt to new technologies quickly, a majority of them does not. Observing my parents and other people around me who have struggles learning how to type, I want to build something that can help them with the learning curve and help them type faster.

What it does

The application let user wear the smart glasses and try to recognize the position of the keyboard. The recognition result works best when the glasses are 20-25cm above the keyboard, which is the recommended reading distance between the eyes and the keyboards. The application then try to detect each row of characters of the keyboard.

How I built it

The application is built on the smart glass Vuzix using Vuforia Engine and Unity.

Challenges I ran into

The tool does not pick up the keyboard image as easy as other images because of the glare reflected from the laptop surface, hence I had to use a workaround.

Accomplishments that I'm proud of

I'm proud of stepping outside of my comfort zone and learning Augmented Reality.

What's next for Keyboard AR

The next step of the application will give users a way to input words or sentences, either through voice or character recognition of another text, then highlight the keys on the keyboard as user types. Ideally, there will be a dataset of keyboards to enhance the analyzation of different layouts and types of keyboards.

Built With

Share this project: