Baymod

Inspiration

In an increasingly digital world, the online safety of children has never been more important. Platforms like Roblox and Discord have become communities of people, sharing common interests, but unfortunately, these platforms also attract individuals with malicious intent. Predators lurk in the shadows of these platforms, exploiting the naivete of children and sending inappropriate messages that can lead to devastating consequences. This problem exploded during the pandemic, when more young children moved online than ever before, and content moderation efforts have simply failed to keep up. Current tools are limited to keyword filters, inaccurately combatting innocuous conversations and completely missing out on the hundreds of thousands of subtle harmful interactions that lead to real cases of explicit media, abduction, and abuse. To combat this, we built Baymod.

What it does

Baymod is an AI-powered, content-aware moderation tool that accurately moderates large online platforms like Discord and Roblox at scale. Baymod combats grooming, bullying, and other harmful behaviors.

Detection: Baymod uses Large Language Models (LLMs) to analyze community language patterns and historical interactions. Baymod is capable of identifying subtle signs of harmful behavior and understands context to distinguish between harmless banter and genuine risks.
Intervention: Baymod alerts moderators and authorized adults to potential issues in real time, and Baymod also provides real-time support to potential victims, offering resources and guidance while issuing warnings to potential perpetrators, educating them on community guidelines and de-escalating situations.
Scalability: Baymod is built to handle large communities, making it ideal for platforms with high user traffic. Baymod uses Retrieval-Augmented Generation (RAG) on a dataset of relevant research papers to stay updated with evolving moderation standards, and it's even customizable for different communities, ensuring its responses align with specific community norms.

How we built it

We built Baymod using the Discord API to build a Discord bot interface for an interactive and mod-friendly interface in the Discord platform itself. For our LLM, we used Kindo API in combination with Langchain to add advanced language capabilities to the bot. By leveraging a Llama3 model through Kindo, Baymod is capable of understanding and responding to nuanced content within chat interactions.

To enhance the bot's moderation capabilities, we implemented Retrieval-Augmented Generation (RAG), which allows Baymod to access and reference research papers detailing moderation behaviors and linguistic tendences of groomers. The RAG setup with LangChain enables the bot to perform tasks such as identifying harmful content, flagging violations, and responding appropriately based on established policies. It also allows Baymod to be more customizable and generalizable.

Finally, we again leveraged the Llama model through Kindo to provide a chat solution for identified victims, helping victims identify that they are in a harmful solution and helping them find relevant and good resources. The combination of these technologies makes Baymod a robust solution for maintaining safer and more inclusive online communities on Discord and other large platforms.

Challenges we ran into

One challenge we ran into was correctly integrating Kindo API with Langchain, which we solved by extended Langchain's base LLM class to wrap calls to the Kindo API, which is the same format as OpenAI's API. We also ran into challenges doing embeddings for RAG, which ended up being one of the more complicated parts of the project that we had to debug.

Accomplishments that we're proud of

We reached our project MVP in <12 hours! This is the fastest any of our team members have ever worked in a hackathon, so we're really proud of being able to collaborate and ship code so quickly.

What we learned

We learned a lot about how to use LLMs to build a product. Specifically, we learned how to leverage AI-as-a-service platforms like Kindo API and common LLM tools like Langchain.