WhoDis App

interface

Inspiration - We were inspired to use voice technology to bring the world one step further to a personal assistant AI. With advances in voice recognition and speech to text conversion we are confident that our application and its peers will see real world results if not today then in the near future.

What it does -

Basic Concept – A One-time-pad code is sent via SMS to be verified vocally. Furthermore we offer an additional layer of security by confirming the user through speaker recognition by using Microsoft Azure.

How we did it

1) We started by making the user interface of the app and hardcoded it using Flutter. 2) The user then downloads the app and registers themselves within our Postgres database. 3) Every time a user registers, the database checks all already existing records and updates the record if it already exists or creates a new record. Each record has a user_id, which use to keep track of our user database. 4) Whenever the user wants to log in any website or known URL, they open our app. The app calls the server and the server messages the OTP using Twilio, which is a text messaging API. 5) On receiving the OTP which is in form of a code, the user is asked to speak the code aloud, and the app simultaneously records the audio and which is when Azure's speaker authentication comes into place. If both the voice and the OTP code match, the user is authenticated into the system, otherwise they get 2 more times to try, after which they are rejected and have to start a new session.

Challenges we ran into - Starting with Android Studio for a recording app, we faced some compatibility issues and the app kept on crashing, so we shifted to Dart, and based our recording app on it.

Another Challenge that we faced was with server-side scripting and database connection, we were using basic forms to get user information using HTML5, so we started using frameworks like flutter and flask to help with server and database documentation.

We also faced problems to connect Amazon Alexa and google home to the wifi, we tried using Connectify, to create a private network, but still, the devices didn't connect, so we had to drop the idea of using them altogether.

We also planned to use google home and Alexa for speaker recognition, but they didn't allow us to pull the audio file which was the prime requirement of our idea, this was when we started using python to convert audio to text and vice versa.

Accomplishments that we're proud of

We are proud to have at least a barebones configuration working, even if it's not on every platform we originally set out for. Our flutter mobile app has also reached a significant milestone being able to record audio and interface with our backend.

What we learned

Don't spread your team thin attempting all your original ideals. It was only toward the end of the hackathon we decided to first come up with the MVP and build upon it from there.