HackUMBC Friday AI

Inspiration

As we were brainstorming for the project, we considered various ideas from motion capture programs to virtual assistants. We valued the complexity of our code, the creativity to produce a new product, and the utility of our program to a wide range of people.

What it does

Ultimately, we decided to encapsulate the idea of a virtual assistant while expanding to greater capabilities. Our virtual assistant, Friday, takes user voice input and is capable of answering everyday questions just like any other virtual assistant. However, we extended it's capability using the Gemini api, which enables Friday to navigate and perform tasks on the user's desktop.

How we built it

The overall program takes in vocal inputs, listening for "Friday" as well as general actions verbs such as "open" or "search." Our primary tool for the creation of Friday was Gemini which enabled the user inputted command to execute.

Challenges we ran into

The development process was demanding. It took multiple trial and errors for the program to take in clear and recognizable vocal inputs. The interaction with the machine learning capabilities of Gemini along with the voice recognition software was hard to combine into a cohesive product considering all the different aspects and dimensions of different webpages.

Accomplishments that we're proud of

Though, through all the setbacks, we were able to create a functional virtual assistant. It was promising to see the end product but also to fix all the minor setbacks we had by testing then rewriting our code.

What we learned

The development of our code as well as the various sponsor workshops helped develop our ability to generate new ideas, test code, and develop skills in different languages, but most importantly, it helped us learn how to overcome obstacles and setbacks regardless of how many times we may fail.

What's next for Friday AI

The current rendition was clearly limited in it's capabilities due to our short time frame to produce the virtual assistance. Our current program was a proof of concept that given more time could develop to be used more effectively in the real world for various applications. If given more time, we would improve the precision of the current features to be accurate in all test measures. In addition, we would also include some greater features to expand on the potential of the generative AI source.

Built With

• python