Adjutant

Demo is available at http://adjutant.tech/

Siri and Google Assistant brought voice control to mobile phones – we're bringing it to every website owner that wants to use it with an easy to embed JS library.

How it works

You define some voice actions, embed the library, and then your users can control your site using their voices. Also, you gain valuable data you can analyse with our custom ML models – right now it infers the user's gender and their emotional state, more to come. Feel free to swap out the model for your own! We also built a simple business dashboard to visualise this data, or you can add it to your web analytics suite.

Applications

Our demo Chrome Extension works for navigating on Medium. One useful application for all kinds of websites is gathering feedback from users – it is so much easier to just say one or two sentences about what you think e.g. about the design instead of filling out lengthy feedback forms.

Technology

Backend

  • app.py - main flask application, methods for recognizing commands and statistics for dashboard
  • model.json - the description of Keras ML model which we used to do Speech Mood and Gender analysis
  • models.py - database model for Feedback entity
  • mydialogflow.py - API methods to do command classification via Google Dialogflow
  • text_sentiment.py - ML model for text sentiment classification
  • transcription.py - API methods for calling Google Cloud Speech for transcribing
  • voice_sentiment.py - Deep Learning model for speech mood and gender analysis

Dashboard

The dashboard is built as a single-page application using React and some Ant Design components.

Built With

Share this project:
×

Updates