Vocalize

LOGO
APP logo

Inspiration

Our inspiration is based upon the way Stephen Hawking, despite his disabilities, was able to access his computer by using a unique user interface that was based upon a small sensor that is activated by a muscle in his cheek. His profound name has always brought about a question in our minds" how can physically disabled people use their computer much more efficiently?'. Our project is called 'Vocalize' it is a program that enables everyone and anyone to use computers just by using voice commands.

What it does

Our program uses Speech Recognition to control the mouse and keyboard, just by saying out the commands loud you perform the Magic. That means one can: 1) Directly open and search on websites. For e.g'' Open Instagram" or "Search for Cats on Youtube" 2) Control the pointer of the mouse and make it point/go anywhere on the screen unlike any other Voice Speech Recognition system using a Grid linear map. Along with Scroll up and down by saying "Scroll up" / "Scroll down, Left-click, Right-Click, Double-click, and other simple mouse functions. 3) Perform all keyboard combinations shortcuts through speech. It also allows users to type short speech to a text editor. 4) Can make random conversations, such as random jokes to make your life easier

How we built it

PYTHON PYTHON library #1 PYTHON library #2 . . . 4) PYTHON library #10000000000 What makes our project more interesting is that we can point the mouse anywhere on the screen by just saying out coordinates on the screen. We divided the screen into a 4*5 Matrix / linear Map. So if we wanted to take our mouse-pointer to go to the top right corner of the screen we would just say (1,5).

Challenges we ran into

We needed a way to move the mouse around the screen efficiently without making it ridiculous. Besides, none of us had knowledge about language processing nor knew what API was. Bringing all small parts of each member together is also a real challenge.

Accomplishments that we're proud of

When we first thought about our topic we were so confused and almost thought it was impossible to make. We took several advices from the mentors, they too assumed it was a difficult task and we were suggested to use Voice Flow, Natural Language API, Wrapper Programs, OCR, etc. However, after like hours of research, we find out we can just use Python and its SpeechRecognition library.

What we learned

We have learned to use Google Cloud Platform Speech to Text, Natural Languages, know how to use several new python libraries and how to merge several files into one efficiently.

What's next for Vocalize

1)We want to make Vocalize a browser extension so it can be used by web developers who want to make their website much more accessible by people with physical disabilities.(Business Opportunities) 2) We want to make our pointer-system more accurate, which can be done by increasing the No. of gridlines 3) Explore new ways the user can interact with the system such as,eye-movement using eye-tracker software to suggest where the user is looking at the screen, mind-control technology, etc... 4) We want to get a much faster voice assistant which can be done using real-time WebSocket.