Pioneer

Inspiration

We were inspired to create a service to aid browsing for specific use cases. Whether you are cooking, gardening, or eating, you want to browse - but you can't type or click. No one wants a dirty keyboard, and no one wants to stop what they're doing and divert their attention to their laptop. Also, Pioneer is useful for those who have disabilities and may not be able to use a keyboard or mouse. We believe in maximizing accessibility for people of all backgrounds.

Some of our influences are Cortana, Siri, Alexa, and Google Home.

What it does

Pioneer is a Chrome extension that uses speech-to-text for voice commands. You can get Pioneer's attention by saying "Hey Chrome, " followed by your request. Your request can be a wide range of browsing functions.

For example, let's say you want to find a recipe and save it. You can say "Hey Chrome, go to Google. Hey Chrome, search 'how to make chicken parmesan'." Let's say the second link looks the most appealing. You can say "Hey Chrome, click on the second link." You found your recipe, now let's screenshot it. Say "Hey Chrome, take a screenshot." Pioneer saves the screenshot with a timestamp in the filename, so you can find it easily.

Pioneer has numerous other functions that enable you to do anything you want on the internet, with your voice.

How we built it

We built the speech recognizer using an API from Google Cloud and the string from the API goes through a decision tree and the tree determines what Pioneer needs to do and the function is able to perform an action on the browser using Selenium.

Challenges we ran into

We started TAMUhack with another project. After attempting to create an innovative solution numerous times, we were stumped. As midnight approached, our team decided to switch projects completely. This added pressure because we had less time to finish Pioneer.

Instead of trying to squeeze something ineffective out of our first idea, we didn't let the pressure get to us and calmly shifted our focus. Giving up on over 9 hours of work isn't easy, but our team worked together to avoid frustration and move on.

Accomplishments that we're proud of

None of us had previous experience with the Google Cloud API. On the spot, we learned the ropes and quickly familiarized ourselves. We were also able to delegate tasks to different members of our team; two different people worked on two different files, and the rest of our team floated between tasks, offering advice and suggestions to extend the functionality of Pioneer.

We are also proud of the hard work we put into this project. All of us stayed alert until 3 AM, and were up early in the morning to get back to coding.