As an engineer at the University of Pittsburgh, Human Engineering Research Labs (HERL) an institution that builds assistive technologies for disabled people I came across a lot of disabled people who have ideas and views that they wanted to express and share with the world, but due to technical difficulties they were not able to share them, they have a different perspective about the world, many things that may be simple for us can be difficult for them, I wanted to build something that will allow them to easily share information about them, thoughts, ideas, experiences online, thereby allowing others to know them better. This is what inspired me to develop the Voicemake application

What it does

Voicemake allows users of all skill levels and most disabilities to easily build their online profiles and create blog posts on topics of their interest just by using their voice.

How can it help disabled students and faculty

When it comes to education, blogging can be a great tool for improving a student's involvement in a topic of their interest, publishing their writing, and educating others on a particular topic. Since blogs are mostly informal it gives students a voice and takes the pressure off writing. It can also help faculty and teachers to publish resources, assignments, and class events, and other topics being covered in class for parents to look at. It can also allow teachers to review students writing skills and help improve them, by building an online profile and portfolio pages students and teachers can also share their skills and projects with future employers, teachers, and others. Voicemake wants to make this process as easy as possible so that any student or faculty even those with disabilities can easily create blogs and profile pages

How we built it

Microsoft Azure AI technologies played a crucial role in the working of the product, the AI services like Speech to Text was used to collect the user input and convert them to text, the Language Understanding service (LUIS) was used to understand the main intent, the Text to Speech service was used to ask questions and respond to the user. The backend of the system was deployed on Azure Virtual Machines with NGINX as a reverse proxy. The backend runs a Python Flask application with a MySQL database as the primary database. There is a NodeJs based token service used to generate tokens and the static webpage build tool Hugo was used to creating the webpages. The notifications are sent out via SMS using Twilio SMS API, It uses one-time authentication codes to make authentication easy and improve security and as these codes are sent to user's phones it is convenient for people with disabilities to access them and read them out on the web app.

Challenges we ran into

Initially, the speech output would overlap whenever multiple questions were asked, I had to use events to solve the issue, the Hugo build requires config files to be written to specific folders, which were sometimes not available. Some of the names especially non-english ones would return incorrect responses the workaround for this issue was to speak one letter at a time

Accomplishments that we're proud of

I am a very hands-on person and I got a chance to learn about the different Azure services and use them to build something useful that can possibly improve someone's life

What we learned

I learned how cloud technologies like Azure provide APIs and Abstractions that can be used to easily bootstrap projects without worrying about the implementation details, this allows developers to focus on creating value for people as soon as possible, I learned about services like Azure AI, Compute, Storage that have good documentation and can be easily used with a few clicks and really liked how easy it was to get them up and running

What's next for

Make the app more robust Feedback loop: Interact with more disabled users, Allow users to give feedback and suggestions, and use AI to find problems and use cases Improving the UI: Add intent confirmations, Reprompts, Previews, Allow adding images Backend: Make code modular, Improve scalability, Availability, Performance, Security, use Azure CDN for static content to ensure global access with low latency, use event-driven micro-services to provide better decoupling

Built With

Share this project: