AVA - Advanced Virtual Assistant

Backend
Loading
AVA
Frontend
Terminal

Inspiration

The inspiration for AVA (Advanced Virtual Assistant) arose from a desire to create a tool that could have a real impact on people's lives. As individuals passionate about technology, we have always been interested in the concept of virtual assistants, but we often found ourselves frustrated by the limitations of traditional input methods. Typing out queries felt cumbersome and inefficient, especially when there was a faster and more intuitive alternative Available: voice recognition. According to the World Health Organization, there are currently 285 million visually-impaired individuals worldwide, with this number expected to triple by 2050. In a survey conducted by the American Foundation for the Blind, 70% of respondents reported difficulty accessing information on the internet due to accessibility barriers. As a result, we realized that our tool could have a significant impact on visually-impaired individuals, who often face significant obstacles when accessing digital technologies. We envisioned a virtual assistant that could seamlessly integrate into people's lives, providing them with quick and easy access to information without the need for any input. As a team of passionate youth, we developed AVA: Advanced Virtual Assistant, a cutting-edge tool that uses advanced speech recognition and text-to-speech features to provide visually-impaired individuals with access to information across the globe. By prioritizing accessibility and productivity, we aim to create a more inclusive society using AVA where everyone has equal access to opportunities. We believe it's our responsibility to act and create innovative solutions to help this population, and AVA is just the beginning. Overall, our inspiration for AVA came from a deep desire to make a positive impact on people's lives through technology and we hope that AVA will be the first step towards creating a more equal society.

What it does

In today's technologically fast-paced world, time is an incredibly precious commodity. That's where AVA comes in as a lifesaver. It offers a wide range of features that can help you save time and effort, from providing suggestions for almost any question you ask to assisting you with simpler, organizational tasks like making lists and editing your writings. But that’s not all it is for. There’s much more! Being a visual companion, it acts as a personalized friend to its users. With AVA, you can access information more quickly and, in a way, curated to your needs and desires. It's not just a tool, but an essential part of your daily routine, providing reliable assistance that you can depend on no matter what. And its edge due to its ability to be integrated with different software, AVA provides an even better user experience and can be your back wherever you go. If you're looking for a tool that can help you stay organized, informed, and productive, while also providing a friendly and personalized experience, AVA is the key. Try it out and see how it can transform your life for the better.

AVA also has the potential to be used for social good. Its impact is most profound for the visually-impaired community. With AVA's advanced speech recognition and text-to-speech features, blind and visually-impaired individuals can finally experience a new level of independence. They can ask AVA to read out text messages, emails, and other digital documents without the need for any other person's assistance.

Moreover, AVA's impact goes beyond just individual users. Many organizations working for the welfare that individuals can use the assistant to enhance their services. Libraries can offer audio books, which AVA can read out, and educational institutions can use AVA to provide students with an inclusive learning experience and take their learning to the next level.

The impact of AVA can be so profound that it can become a must-have tool for any organization that prioritizes accessibility and inclusivity. AVA is not just a virtual assistant, but a symbol of hope and empowerment for millions of people with visual impairments.

How we built it

For coming with a software which serves an all-rounded purpose and while not compromising time and efficiency, our main target was to look around for the possible spheres of research to implement all this all at one place. We decided to come up with a software sufficient enough for us to strike and continue a generic conversation to an intellectual talk between the user and the model. Hence, being a team of four, we divided our front-end development and back-end development equally amongst our members. While two of us focused on the back-end, the other two constantly worked to set up a stage for them by designing the front-end of our project.

Our key target was a smooth user experience with full satisfaction. For that, the most important thing was to catch the viewer’s attention and keep it engaged throughout the conversation to mark an impact on their mindsets for them to use it again and again for productive purposes. While it can be significantly used for malicious intents, we tried to ensure it does not derogate itself and ruin the user’s experience in any unknown manner.

To begin with, the front-end, as the name suggests, was the spotlight of our project. Hence, coming up with a fully functional design and software enticing enough to catch user’s attention was a major task. We aimed to create a dark, simplistic design with attention to detail, including animations and a tonality sound enough to convey our message. We designed a web application, defining the starting point, layout, and integrating various components to make it functional. It was important to ensure the application's responsiveness, allowing it to work seamlessly across different platforms and devices. Overall, we worked hard to create a software that would fulfill our expectations and provide a positive experience for our users.

Followed by this came into account the most important aspect which distinguished our model and provided it a unique standpoint - the face of AVA. We wanted to design our own specific modular face using threeJS and hence one of our developers focused on that part.

Furthermore, the back-end part was the backbone to the entire idea summed up above. It was ideally divided into three parts - 1) Taking in inputs as speech (or predefined audio per se). 2) Converting that audio into text 3) Generating suitable audio-based outputs based on the textual input

To achieve this goal, we carved our path using the asphalt of APIs. Since we wanted our outputs to be personalized specific to the requirements of every user, we took to APIs to obtain necessary AI models and used Natural Language Processing and its further deep-rooted intricacies as a bridge to the conversation. For these purposes, we used OpenAI’s Da-Vinci module - which falls under the umbrella of the GPT3.5 model - as our base API. Fundamentally, we designed two softwares to convert speech to text and to reverse the text to speech. The purpose of doing and then collaborating it into one was to retain the precision which could’ve been hampered. As a result, we made more user-centric models which were not heavy for the developer environment as well. Since we used API as the bridge to these, it proved to be much more efficient for us to use it through a mix of both the softwares.

Challenges we ran into

One of the primary challenges we faced was developing a personalized face in threeJS that could synchronize with the output speech. This required extensive experimentation and testing to ensure that the facial movements matched the spoken words in a smooth and seamless manner. Another significant challenge was integrating the front-end HTML, CSS, and JS code with the Python code from the back-end, using the Flask module. This required careful coordination and communication between team members, as well as a thorough understanding of the various components involved. We also had to find a way to obtain a voice that sounded natural and human-like, rather than robotic or artificial. This involved researching and experimenting with different text-to-speech tools and techniques until we found the one that best suited our needs. Additionally, we needed to ensure that the output speech was rendered and played back without any significant latency or errors, and that any code within the output was properly identified and handled accordingly.

Finally, we also needed to provide the option to either print out the output text or read it aloud in the form of an audio, depending on the user's preference. This required careful consideration of the various audio and text output options AVAilable, and extensive testing to ensure that everything worked seamlessly together.

Through hard work, perseverance, and collaboration, we were able to overcome these challenges and deliver a software that we are proud of. Our experience has taught us that the journey is just as important as the destination, and that tackling challenges head-on can lead to growth, learning, and success.

Accomplishments that we're proud of

Our team experienced a sense of pride and accomplishment as we celebrated several small victories that we achieved along the way. With each milestone, we not only gained new knowledge but also felt a sense of satisfaction that propelled us towards our ultimate goal. From envisioning success to nearly achieving it, each accomplishment motivated us to aim higher and continue progressing.

Our achievements included a precision rate of 98.05% for our virtual assistant, which outperformed our closest competitor, a voice assistant, whose efficiency level was only 93%. Additionally, we successfully engaged diversified ideas using various frameworks and mechanisms. Initially, we knew little about integrating code in different languages, but now we are confident in our ability to combine codes and scripts from different languages and models into a parent code.

Moreover, we were able to finish our project within a 24-hour time frame with constant support and motivation from each other. This was an unexpected accomplishment as we had thought that it would take longer to complete. As our first hackathon, this experience has proven to be a valuable learning opportunity that has prepared us for future challenges.

What we learned

A significant reason, among many others like genetics and other external factors, as to why and how people experience addiction has to do with the level of disconnectedness they experience with their loved ones and other people. We decided to create an app that provides recovering opioid users with the help they rightfully deserve. Our app includes a Resource map, emergency contacts of the user, medical logs and more. Six of us with unique skillets and different ideas regarding each feature learned how to get together and work for a better cause. With every other brilliant feature we pursued, we encountered new errors, bugs, and issues from which we had to bounce back with new creative solutions. With the time constraint along with being stuffed with classes and internships, every single one of us learned to push ourselves to the limit and beyond to make this vision come true.

What's next for AVA - Advanced Virtual Assistant

The AVA (Audio Visual Accessibility) project is committed to constant innovation and improvement, and we plan to scale it further in the coming months. Our aim is to make it more mobile and AVAilable on any device, website, or platform to cater to the needs of all users.

While we have made significant progress in developing AVA's individuality, we recognize that there is still room for improvement. One area where we plan to focus is creating a graphically interactive model that is geometrically simplified and can be controlled using AVAilable controls. Additionally, we aim to configure AVA to cater to the needs of each individual user, including those who are visually challenged. We plan to bring in features that favor the ease of use for visually impaired patients.

Furthermore, we aim to make AVA accessible to a broad range of linguistic groups and communities by making it AVAilable in different languages. Our goal is to eliminate language barriers so that users across the globe can access our software and improve their lives.

In conclusion, AVA is a dynamic project that will continue to evolve to meet the diverse needs of its users. Our commitment to innovation and improvement ensures that AVA will remain relevant and useful for years to come.