modassist-ai

AI-powered Reddit moderation assistant that detects spam, scams, and suspicious content in real time using Gemini AI.

Comment

Inspiration

Reddit moderators deal with an overwhelming number of spam posts, scams, fake giveaways, malicious promotions, and abusive content every day. Most moderation still requires manual review, which becomes difficult as communities grow.

We wanted to build an AI-powered moderation assistant that could help moderators make faster and smarter decisions directly inside Reddit. Our goal was not to replace moderators, but to create an intelligent co-pilot that reduces workload and improves moderation efficiency in real time.

That idea became ModAssist AI.

What it does

ModAssist AI is an AI-powered Reddit moderation assistant built using Devvit and Gemini AI.

Whenever a new post is submitted in a subreddit:

  1. Devvit triggers automatically detect the post
  2. Gemini AI analyzes the content
  3. The system classifies the post as:
    • APPROVE
    • REMOVE
  4. The moderation result is stored and displayed in a moderation dashboard
  5. Moderators can instantly review AI recommendations and take action

The app analyzes:

  • post titles
  • body text
  • suspicious promotional patterns
  • scam indicators
  • malicious links
  • spam behavior

The moderation dashboard includes:

  • categorized moderation tabs
  • confidence scores
  • AI reasoning
  • analytics
  • one-click moderation actions
  • real-time moderation feed

Example AI output:

{
  "action": "REMOVE",
  "confidence": 99,
  "reason": "Potential cryptocurrency scam and suspicious promotion detected."
}

How we built it

Frontend

We built the UI using:

  • React
  • Tailwind CSS
  • TypeScript
  • Devvit WebView

The interface includes:

  • moderation dashboard
  • moderation categories
  • analytics cards
  • animated toast notifications
  • loading skeletons
  • responsive UI components

Backend

The backend was built using:

  • Devvit server
  • Hono
  • Redis
  • Gemini AI API

We used Devvit triggers to listen for Reddit post submissions in real time.

Moderation flow:

New Reddit Post -> Devvit Trigger -> Gemini AI Analysis -> Moderation Decision -> Redis Storage -> Moderator Dashboard

The moderation system stores AI analysis results, moderation states, analytics data, and moderator actions for live dashboard updates.

Challenges we ran into

One of the biggest challenges was understanding how Devvit applications integrate with Reddit moderation workflows and real-time triggers.

Some technical challenges included:

  • handling Devvit trigger events correctly
  • integrating Gemini AI with Reddit post data
  • managing Redis state efficiently
  • handling AI quota and model compatibility issues
  • building moderation workflows inside a WebView environment
  • designing a scalable moderation state system
  • reducing false positives from AI moderation

We also spent significant time improving prompt engineering to ensure moderation explanations were meaningful and useful for moderators.

Accomplishments that we're proud of

We are proud that we successfully built:

  • a fully working Devvit moderation app
  • real-time AI moderation triggers
  • Gemini AI integration
  • a responsive moderation dashboard
  • categorized moderation states
  • moderation analytics
  • Reddit-native moderator workflows

The project evolved from a simple prototype into a much more complete moderation platform during development.

We are especially proud that moderators can now review suspicious content almost instantly with AI-generated reasoning and confidence scores.

What we learned

During this project we learned:

how Devvit apps work internally

  • how Reddit moderation systems interact with apps
  • real-time moderation architecture
  • AI prompt engineering for moderation systems
  • Redis-based state management
  • WebView-based application development
  • moderation UX design principles

We also learned that explainability is critical for AI moderation systems. Moderators need clear reasoning and confidence scores before trusting automated recommendations.

What's next for modassist-ai

We plan to continue improving ModAssist AI with:

  • OCR-based image moderation
  • suspicious URL reputation analysis
  • AI comment moderation
  • NSFW image detection
  • moderator collaboration tools
  • automatic moderation actions
  • user trust scoring
  • Supabase integration
  • advanced analytics dashboards
  • feedback learning loops for moderation accuracy

Our long-term vision is to create an intelligent AI moderation co-pilot that helps Reddit communities stay safer while significantly reducing moderator workload.

Built With

  • ai-prompt-engineering
  • devvit
  • devvitwebview
  • gemini-ai-api
  • hono
  • node.js
  • react
  • real-time-event-triggers
  • reddit-developer-platform
  • redis
  • responsive-ui-design
  • rest-apis
  • tailwindcss
  • typescript
  • vite
  • webview-based-architecture
Share this project:

Updates

Submission history