As covid struck, we all had to move back to our hometowns, from where we have been attending online classes for more than a year now. After experimenting with multiple platforms of online conferencing, Gmeet undoubtedly proved to be one of the best platforms to attend classes, meet up to discuss projects, or just generally hang out.
Gmeet brought ease of access, allowing anyone with a google account to use its video conferencing service, but it lacked in functionality. There was no way to take attendance, no functionality to generate a fillable form and no feature to sum-up the crucial points. Apart from these basic features, it also lacks in more advanced features, such as real-time chatbots and others, which are provided by its competitors.
We wanted our favorite video conferencing app to have better features. Hence, the idea of 'Hirav' was born. A bot that gives Gmeet the features it deserves and also makes a boring meeting a little bit more interesting.
What it does
Hirav is a Google Meet bot that adds a multitude of features to a Gmeet online meeting It can perform 10 main functions that are as follows:
- A chatbot to use with FAQs, greetings, and much more : A crucial feature in this bot is the ChatBot. It is a trained Deep Learning model that can be taught to answer the necessary questions. In our prototype, we trained it to greet people and also to answer FAQs related to a Hackathon. But with the right dataset, anything is possible and it can answer a lot more questions. The chatbot can be activated by using a ‘/bot’ command before any question.
- Providing with a survey/attendance form: The organizers might not want a direct list of the usernames. Instead, they might want to take attendance using a Google form. Hirav can do the task for you. It will create a form with the name and Id/Registration Number as the input fields and will put up the link in the chatbox.
- Checking for inappropriate/foul language: It checks if the language used in the chat section of the meeting is inappropriate/foul. After checking it puts up a message asking the person not to use foul language. It also adds the username of the person to a pdf which will be saved on the teacher’s/organizer’s device.
- Making a list of all attendees: The bot can be invoked by putting up a command ‘/attendance’ which will read the entire list of all the attendees currently in the meeting. It then adds this list to the same pdf as mentioned above.
- ** Adding important/crucial points into a MOM:** Using the command ‘m/’ and then writing whatever point you want to add will add that point to the pdf created. For example, the teacher wants to note that chapter 2 has been completed. He/She can just write ‘m/Chapter 2 completed’ and just like that, it will be added to the pdf.
- Formation of the PDF: If any one of the previous 3 functions, the MOM, Attendance List or Foul Language features are invoked, a pdf will be created containing the necessary details. This pdf will be saved in a local directory of the computer which invokes the bot.
- Taking screenshots: Another great feature is that the bot can take screenshots on command. The command ‘/ss’ takes a screenshot of the current screen as seen by the teacher/organizer and not only saves it locally, but also uploads the file to Google Drive and sends the link in the chat.
- Providing with the name of a random attendee: Online classes are usually just the teacher talking, with maybe one or two students answering the questions. If the teacher wants a different student from the list of attendees but doesn’t want to scroll and pick, they can use a ‘/random’ command which will output the name of a random person from the list of attendees. (To the student community: We are very very sorry!)
- Generating an inspirational quote: An added feature to the bot is that it sends a famous quote along with the author name whenever you type ‘/quote’
- Providing with a new word: Similar to getting a quote, it can also generate new words and put them in the chat along with the meaning. It can be accessed using ‘/newword’ command. This will surely help people to increase their vocabulary.
Each function has a specific keyword provided to it. Upon entering the keyword in the chatbox the bot will read the command and execute it in seconds, either giving the required result or an acknowledgment in case of creating minutes of the meeting or taking attendance.
How we built it
Hirav is coded in python, with automation and deep learning being at the heart of its code.
The bot works using a specific list of commands. These commands when typed inside the google meet chatbox execute their respective code. Each function is built to automate. The functions use the web environment to extract the required data, process it and give the necessary output.
The Automated chatbot, which is also a vital component of Hirav, works with a Deep Neural Network model using tensor flow on the backend. A JSON file works as the data file, for training the DNN model. The weights of this trained model are used for achieving the best response to someone's question.
Challenges we ran into
Automating something which lacks so many key functionalities itself created most of the issues. The biggest challenges we faced were dealing with the weird UI of the chat heads on the google meet environment, dealing with the chatbot component, and finally combining the entire code.
The UI element of chat heads on google meet chat made it challenging to extract the commands from the chat. The change in the web address of the chat components was not uniform. This lead to the code being stuck and not being able to read every command.
The chatbot was the most problematic as one improper training of the DNN model could create an issue and make the bot give improper responses. We had to monitor the training so that the training resulted in the correct weights for the chatbot.
The code initially was in a lot of bits and pieces since everyone was building individual features. Combining everything to work together and run as a single unified bot took a lot of patience and teamwork. However, each and every member worked tirelessly to finally complete the project.
Accomplishments that we're proud of
We had never done such a big development project in python. To achieve so much in our first python-dev project is something that made us immensely proud. After hours of integrating all the modules, libraries, and python script, it was all smiles when the entire thing worked without a hitch.
It may sound frequent but solving our biggest challenge was undoubtedly an achievement. The non-uniform change in the paths of the chat heads had made it challenging to read consecutive commands in some scenarios. After running hours of manual debugging, we were able to figure out an algorithm, that made sure each command was read and executed perfectly.
Apart from the technical accomplishments, we are immensely proud of how well our team worked and performed together, throughout the entire duration. Everyone helped each other solving doubts and clearing concepts in their part of the code.
What we learned
Preparing this project in a couple of days proved to be a refreshing break from all the riveting college work. We all had previously worked with python and did know the basics well, but making this project, we were pushed to the limits of our knowledge. Our debugging skills, teamwork, and confidence as developers have increased.
Automation of website/webpages is something that has gone uphill in the past few years. With the introduction to control web-elements through code scripts it has become possible to automate a tedious process on websites. Exploring the fields of automation we were able to learn and work with new and upcoming technology.
This bot let us test two of the most advanced tech-stack in the modern era. We were able to do automation adding several features to a simple system and also implemented a chatbot using deep learning!
What's next for Hirav - The smart companion bot for your G-Meet
Hirav is just our first try on a python-dev project, and it is functioning really well. Its given commands could prove momentous towards a companion on google meet, as we have also emphasized on its lack of some basic features many times. Hirav still has a long way to go when it comes to adding features and optimization. The first version that we have built has been tailored to education professionals along with some official scenarios. This bot still lacks some major features when it comes to conducting massive online events and some fun components.
Our vision with Hirav is to make it a perfect companion for all kinds of users on google meet. We plan on adding components like some games, music, and video features. Making the chatbot more interactive and increasing the speed of response of the bot would be our biggest priority.
We would also want to add a live note-making feature in the future. We can think of adding an authentication service and link it to the Google Calendar to schedule and join meetings referring to the calendar. Finally, our ultimate goal would be to compile everything and make it a chrome extension or a web app such that anyone can use it to its full potential.
Stay tuned for Hirav 2.0. :)