-
-
The Logo
SpeakSafe
Inspiration
In our time on the internet, we have found that while people are capable of great loving, they are also capable of great hating. Hate speech can cause great damage to the groups it is levied against. While hate speech moderation is common for text chats, it is rare for voice chats. That is where our project comes in: a discord bot which moderates voice chat and kicks users who say hate speech.
What it does
SpeakSafe works as a discord bot that constantly listens to and parses the speech said in a voice call. This speech is then transcribed to text by the bot. The text is then sent to our machine learning language model hosted in the cloud. The neural network then decides whether the text is hateful against marginalized groups or not. The discord bot then uses this information to either leave the person be or take action and kick them from the call or server. Additionally, users can enter custom phrases they want banned as a safety measure, through a custom website.
How we built it
First, we constructed a discord bot, the Discord.js framework, which interfaces with the Discord API, allowing us to perform actions in Discord. When activated, this bot constantly monitors audio input from every participant in a voice call, and passes that audio stream to the Wit.AI API for speech to text transcription. Using the transcribed text, we first compare this to a user-defined list of banned words stored on a MongoDB database instance. Users can add to this blacklist by inputting phrases into a custom website built using Flask. If the transcribed text clears the blacklist, then we pass the text to a Machine Learning Language Model we set up using Google Cloud Run. We interface with this Google Cloud instance as an API endpoint using a GET request. The model we use specifically, is the Pysentimiento Hate Speech Detection model, from Huggingface. This model will give a percentage of how sure it is that the given text is hate speech. The Discord Bot then receives this info. If the text did not pass the blacklist check or the language model check, we then kick the user from the voice call, and notify the server.
Challenges we ran into
We originally had an issue hosting the ML model on Google Cloud, so we decided to make a web page using Flask to host the ML model, then host that website on Google Cloud.
Accomplishments that we're proud of
The bot does a great job of transcribing the speakers text, catching both the whole text and without many or any mistakes. With this accurate transcription, the ML model is also very good at classifying the text, with it rarely making false positives or missing true cases of hate speech
What we learned
We learned how to deploy API's to google cloud. We also learned how to post and get data from a MongoDB when using it for our front end and back end website.
What's next for SpeakSafe
We wish to make this bot for Slack, which is more business orientated. The bot is only able to work in one voice chat, however discord servers can have multiple different voice chats. We would like to make the bot more versatile, working in multiple voice chats and in group chats, not just servers. Also with a better ML model, we could get the bot to be more accurate. Being able to transcribe and analyze speech in different languages would also be part of our stretch goals
Log in or sign up for Devpost to join the conversation.