GSTBot

Frontend application
Arduino coding
Nodemcu webserver host
sms frontend

Inspiration

Ironman's JARVIS, an intelligent chatbot which had the ability to analyse and act on any data that was provided to it. Our idea was to create an interactive application which could take inputs in any form, audio, video or text. Our product could be easily injected into any environment and be easily customised.

What it does

The bot can accept input in the form of Text, Voice or Gestures. Each of these inputs is processed and appropriate responses are provided. This Bot has the ability to the manipulate the lighting of a home, Play music and provide the weather information.

How I built itbold

With the help of various resources such as APIs for Speech recognition, Text Recognition & Analysis, Gesture Control, Weather information and Music encoding applications provided by Google, Microsoft, OpenCV and VLC media. Our entire control module was hosted on a web server built using the ESP8266 Wi-Fi module. Using the Ngrok tunnel software we were able to connect to the host.

Challenges I ran into

Initially, we had a slow start with Microsoft Cognitive Services, because we were not familiar with C#. Invoking a python function with C# was also a challenging task, but we managed to do it eventually. Integrating the NodeMCU with the web server in order to allow remote control of the physical devices.

Accomplishments that I'm proud of

Building a chatbot with the help of Microsoft API, the ability to analyse text and parse it to the backend and prove appropriate response to the user, integration of voice and gesture recognition in order to control the various parameters.

What I learned

Building chatbots using Microsoft-Cognitive-Services API, learning to use Bot Emulator to test cases or use case scenarios of the project, integrating python code with Visual studio to process requests.

What's next for GST bot

The bot can be made intelligent by using deep learning algorithms and database management in order to provide a more local solution and possibly more accurate responses. This will allow the bot to easily adapt and integrate into any new environment it is introduced to.