My grandpa has arthritis. Meaning as he ages his bones get more tender. Intense swelling might show up if he uses his hands or limbs, therefore keeping him away from technology. Patients with arthritis in their hands have a hard time typing and clicking the mouse and keyboard. This caused me to create such a device that would enable people, who have arthritis in their hands and are disabled, to use technology. Another inspiration is, after I sprained my hand, It was very hard doing every day items. I could not pick up glasses or cups, couldn't open the door, and especially couldn't use the computer.

What it does

SoftClick is a smart mouse and smart keyboard interface. Which allows you to control your mouse without the use of clicking. Example, you can click on an item without physically clicking the button. It also enables you to voice type. Which is done by a voice recognition machine that I developed using Google Speech API.

How I built it

Many different components, libraries, and jar files came when creating this project. I used an algorithm that determines if a mouse is idle in a certain threshold. If so, two separate guis would popup. One would be stating "left," for left click and the other "right," for right click. Hovering over a certain gui would enable the certain action stated. Then the algorithm will determine if there is a text area or not. If so it will start the voice recognition software. For this software I used the Google Speech SDK(api). Afterwards, the program would determine the certain keyboard movements that must be made in order to write out the specific command.

Challenges I ran into

The biggest challenge. Was installing the google speech api into eclipse. It also has hundreds of more methods that I had to research about in order to know what was happening. Another challenge I ran into was that the macbook would disable the accessibility feature for other sources to control the mouse and keyboard. Therefore, I had to figure out a way to let the macbook accept the source and let it hacks its mouse and keyboard.

Accomplishments that I'm proud of

I was really proud of that I didn't give up on the voice recognition software. I had search multiple videos and write ton of code in order to finally get the voice recognition software to work. s

What I learned

How to implement certain API and SDK. The certain methods that were in the Google Speech API and how to use them in your code. The most important thing I learned was patience. It took hours to finally get a little part of the speech recognition software to work.

What's next for SoftClick

Right now SoftClick lifts the burden of peoples hand for using technology. There are many people around the world, example millenarians, without hands to write. Therefore, in the near future I would be creating a device that could sense what command you are trying to perform with the scan of your retina. Depending on if it is left wink or right wink it would perform a left click or right click. Afterwards, I will research about EEG device which sense electron pulses throughout your brain. Using the Arduino Uno and the EEG device you could potentially could it to determine if you are trying to type the letter "A", "B", "C", etc.


Built With

Share this project: