Inspiration

1.7 billion people today have cell phones but no Internet connectivity. A large portion of this demographic further faces socioeconomic challenges such as low incomes, illiteracy, and poor hygienic conditions, leaving them vulnerable to unhealthy lifestyles and malady as they lack information. Several of our team members have been in such environments where a lack of awareness and stigmatisation of mental and sexual health have led to depression and suicides, discrimination (particularly against women), and the continuation of poverty cycles. We built IEve to tackle this issue.

What it does

IEve is a virtual assistant available on phone providing critical information even when users cannot connect to the Internet. A user can call an IEve number and ask a question, particularly for topics of which they lack awareness and/or that may be taboo in their communities. IEve then automatically generates a response to their query using crowd-sourced information that is peer-reviewed and verified by experts. IEve then reads out the answer over the phone so that it is accessible to any type of user.

IEve always ensures high confidence in its answers. If it is not confident, it admits so and automatically opens the question to experts on Einstein. An unanswered question is posted to IEve’s web forums, where users such as doctors, NGOs, and other experts can write responses. Once a response is written, it is peer-reviewed as other experts upvote or downvote it. Once it passes a certain threshold, the response is then sent to selected expert moderators who can then finalise its validity. It is then automatically added to the knowledge base on which IEve’s automated model is trained.

As of now, IEve is specialised in topics on mental and sexual health, which are particularly taboo in countries like India and other parts of the developing world. However, IEve is set up for scalability and will be able to handle any requests as more crowdsourced data is generated.

How we built it

Our system has multiple components mentioned below:

Voice assistant on phone - We use Twilio to provide a way for users to talk with IEve. Twilio handles incoming calls from the user and, as defined by a Node.js script on an Azure VM server, sends the audio stream to Google Cloud’s cutting-edge speech-to-text model for real-time call transcription. The transcribed query from the user is parsed and forwarded to a Q&A server. The response from the Q&A server is then parsed, and the answer is extracted and read out and/or texted to the user using Twilio. If a user calls and an answer is not available, the question is posted to Einstein through a REST API call. The speech assistance makes IEve accessible to all users, even those who are illiterate or unfamiliar with smartphones. Cell phone integration makes critical information available without Internet access.

Q&A - Our query answering system is built using Azure’s state-of-the-art QnA maker. We generate a knowledge base by scraping verified articles and automatically extracting question answer pairs. The knowledge base is extendable by either directly adding question-answer pairs or by adding more articles to scrape information from. Over time the community efforts on Einstein will organically grow a rich knowledge base.

Einstein - The admin panel is built using the MERN stack (MongoDb, ExpressJS, ReactJS, and NodeJS). We followed a microservice based architecture and deployed our front-end application and backend API using Heroku. The QnA model scrapes the approved page on a daily basis to add to the knowledge base.

Challenges we ran into

Our main challenge was integrating all our systems so that all of our applications could seamlessly communicate with each other. We used various APIs including Twilio and Node.js frameworks such as WebSockets and MongoDB as intermediaries that handled communication between systems. We were also challenged initially by the non-robustness of our Azure QnA model, which we addressed through question generation and additional data collection.

Accomplishments that we're proud of

We are quite proud of how we integrated all our systems so that we could leverage the strengths of Azure, Google Cloud, and the Twilio APIs. Our whole application is entirely automatic! Our application is very scalable, and as we hope to help people with information, we are proud that this should increase our scope for impact with time. We are happy that we learned so much in such a short time.

What we learned

We learned a lot about using Azure, Google Cloud, and Twilio and about web development, NLP, and application design.

What's next for IEve: An Organically Improving Voice Assistant for All Information for Everyone

Going forward, we will continue to develop IEve to handle more topics more robustly and aim to promote Einstein and begin crowdsourcing with improved automatic knowledge extraction and updates.

Try it out

Call +1 (205) 579-8843 to see IEve in action Check out Einstein at: https://oosharma.github.io/einstein/

Built With

Share this project:

Updates