ViDo | Devpost

Source: hd-wallpapersdownload.com

ViDo - Voice in Data out, a name inspired from the hackathon itself.

Inspiration

During our journey from Delhi to Surat, the internet in our mobile phones was not working, we were completely disconnected from the world. So we thought how cool would it be if we develop a mobile service that answers our queries through phone, and even extended it to even answer some queries in Hindi.

What it does

The basic service that our system provides is - People call on our phone number and ask their query in English or Hindi(this can be any question, generally fact based), once the response is calculated it is sent to their mobile number through SMS. We use various methods to cluster requests into various categories using natural language processing and machine learning and make the query to the right API like Watson, Wolfram Alpha, Duck Duck Go and google maps etc. to get the best answer for the user. One can also try to go to (http://vidos.mybluemix.net/q/query) with a query and get a detailed answer, eg: (http://vidos.mybluemix.net/q/prime%20mininster%20of%20india)

How we built it

We used a lot of technologies to build this. First of all to receive a call from the general public we needed a public phone number and an API which transfers the voice query from the phone call to our web-server. Once we received the audio query, we first converts it to the default language English(en-US), then once we have the text for the query the possibility to do with it are endless. Like parsing it and using various APIs like IBM Watson, Alchemy, Wolfram Alpha, Google Search, DuckDuckGo etc.

Challenges we ran into

When designing such a large system, it is obvious that there will be a lot of problems that we encounter in development phase. Like the IP being blacklisted on google after the trial period of the developer account was expired and showing captcha, every time we need to translate something, so we faced a lot of problems in automating the testing process. Another main challenge that we faced was when we were hosting our app in the cloud, we couldn't install a few things that we needed to convert audio files, send them etc, to solve this we planned to use our own RasPi server to process the speech input and parse and generate response on Bluemix, the best thing being that the RasPi can be configured for any language and used locally, like here we used Hindi, and it can be configured for something like French or Spanish. Also incorporating Hindi input and sending SMS reply back in Hindi needed a lot of research, as we have never worked on such kind of thing before, like working with audio codecs. Another problem that we faced like the audio file was not available for a few minutes after the call, so we needed to spawn a different thread for each response while managing load on the server, to wait for audio to be available. But as the speaker from TCS said 'Never Quit' and we kept working on solving the problems and finally managed to develop a working prototype, overnight.

Accomplishments that we are proud of

With Zero experience in Audio encoding/decoding/manipulation, we were finally able to develop a system to develop a system which takes queries in Hindi/English and answers them quite accurately.

What We learned

We learned a lot of things today, like audio manipulation(i have said it a couple of time now), various google, wolfram, watson, exotel, APIs, get to know how to solve problems that you never thought can happen, interacted with such awesome mentors, who told us a lot of new things, that we didn't know before.

How we used IBM Bluemix

We hosted our parser and 'Problem Solver' on IBM Bluemix, which takes in the query after been translated to english and calculates the answer. All the data processing and APIs integration was done on Bluemix. We also incorporated other IBM APIs like Watson and Alchemy to get meaningful answers to questions on topics like travel and healthcare.
EDIT: Our API keys have been expired so one might get empty response on questions on healthcare and travel.

Our revenue model

Revenue can be generated by giving advertisements when the user will call our number. We will collect all the data of the users and help advertisers give targeted advertisements. By the advertisements we can cover the cost of the phone calls and make the number toll free.

What's next for ViDo

We have planned a lot of things for ViDo. We are planning to add a make a standalone one click installer for windows which uses windows API to control things in your computer and incorporate few commands like open start, message someone on facebook through browser automation, etc the possibilities are endless. Also we can keep adding many more commands and APIs to make it better, which we cannot do now because of lack of voice training environment, and roaming on our phones.