ModCommand

ModCommand turns your subreddit into a moderation command center. Reports, AI-spam detection, edit-evasion, ban appeals, team workload, and an AI co-investigator — unified in one Reddit-native surface

Comment

Inspiration

Open any moderator's browser and you'll find six tabs: modqueue, modmail, user history, a spreadsheet of bans, an AutoModerator config, and a Discord channel where the other mods are arguing. Reddit gives moderators a fire hose; it doesn't give them a cockpit.

We kept coming back to one observation: every moderation decision is about a user, but every existing tool is organized around an item. A flagged post. A single comment. One modmail thread. The mod is the one stuck mentally joining tables — "wait, isn't this the same person who edited their post after a report last week?" — and that join is exactly where mistakes (and burnout) live.

If you think of a moderator's mental load as a cost function, today's tools optimize for the wrong variable — the cost per item:

$$ E = \sum_{i} C(i) $$

…when what actually drives good decisions is the cost per user:

$$ E^{*} = \sum_{u} C(u), \quad C(u) \ll \sum_{i \in u} C(i) $$

ModCommand started as a hackathon-shaped answer to that frustration: what if the dashboard did the joining for you?

What it does

ModCommand is a unified moderation cockpit that lives inside any subreddit as a single Devvit Web custom post. It bundles five purpose-built dashboards and two cross-cutting AI panels that follow you everywhere.

The five dashboards

Tab Purpose
🗂️ Triage Board Reports, AI flags, and pending high-stakes actions in one Kanban-style queue with claim locks so two mods never collide on the same item.
🧠 AI Sentinel Scores every new post for AI-generated content and auto-reports above your threshold. Adapts the threshold from your team's own past decisions.
✂️ Edit Watch Detects "edit-after-report" evasion — a post is innocent when reported, then quietly rewritten into spam minutes later. Stores a full diff.
📬 Appeal Desk Structured ban-appeal intake via modmail with AI-generated risk summaries — "medium risk: takes ownership but prior evasion."
📊 Workload Wall Per-mod action counts, response times, and a weekly digest cron so the load is visible instead of guessed at.

The two cross-cutting layers

  • 🤖 Mod Copilot — one-click recommendation on any item, anywhere in the app. Verdict + confidence + a chat thread you can interrogate.
  • 🔍 User Dossier — click any u/username from any tab and a panel slides in with the user's full footprint in your sub: account age, recent items with scores and outcomes, evasion count, appeal history, and an AI behavioral summary.

The Dossier is the answer to the original frustration. Item → user pivot in one click, no joining required.

How we built it

Stack at a glance

  • Runtime: Devvit Web — custom posts + server + scheduler + 11 event triggers
  • Server: Hono on the Devvit Node runtime, ~10 REST routes behind a single requireMod gate
  • Client: React + Tailwind, served as an inline iframe (splash.html → dashboard.html)
  • State: Devvit Redis — every signal flows through ~25 typed key builders in src/redis/keys.ts
  • AI: Google Gemini 2.5 Flash-Lite with a SHA-hashed response cache to deduplicate identical content

Architectural decisions worth calling out

  1. One server, one mod gate. Every /api/* route passes through requireMod middleware backed by a 5-minute cached mod list — auth is decided once, not re-litigated per route.
  2. Triggers are the source of truth, not the UI. When Reddit fires onPostSubmit, we score the content, write to the sentinel feed, and update the per-user reverse index before a mod ever opens the dashboard. Opening the UI is just a read.
  3. Signal fusion as a first-class layer. dossier.ts, copilot.ts, and adaptiveThreshold.ts exist to combine signals other modules write — they don't own triggers themselves. This is what lets the Dossier surface evasion history and appeal status in the same panel.
  4. Audit log as connective tissue. A single recordAudit helper writes to one sorted set per sub. The Dossier reads from it. Adaptive Threshold reads from it. Future analytics will too. One write, many readers.
  5. Cache aggressively, invalidate precisely. AI responses (SHA-hashed), mod lists (5 min), dossier payloads (60 s), behavioral summaries (24 h). Each has an explicit invalidation hook tied to the action that would change it.

The Adaptive Threshold algorithm

AI Sentinel doesn't ask you to guess the right threshold — it watches your team's decisions and suggests one. For every AI-scored item your mods approve or remove, we record a sample. We then bucket samples into 5-point bins and find the lowest bin where removal rate crosses 70%:

$$ T = \min \{\, b : \frac{R(b)}{R(b) + A(b)} \geq 0.7,\ \ N(b) \geq 5 \,\} $$

…where $b$ is a 5-point score bin, $R(b)$ is the count of items mods removed in that bin, $A(b)$ is the count they approved, and $N(b) = R(b) + A(b)$ is the total sample count in that bin.

…cold-start guarded at 50 total samples, and the banner only nags when the suggestion diverges from your current setting by more than 2 points. The model doesn't replace your judgment; it surfaces it back to you.

Challenges we ran into

1. The item-vs-user impedance mismatch

Our first sketch had Dossier as a sixth tab. We threw it out — adding a sixth silo to a product whose problem is silos. Rebuilding it as a slide-in panel that lives inside every other tab was the right call but required lifting state through the whole webview tree.

2. Reddit's anti-spam filter fought our demo seeding

Brand-new accounts posting spam-pattern content get auto-banned site-wide before our triggers even see them. We pivoted to a seeding strategy that posts genuinely-innocent content and then edits it post-report — which, conveniently, is exactly the evasion pattern Edit Watch was built to catch. The product caught the demo.

3. Devvit Web iframe identity quirks

getCurrentUsername() returns undefined for non-developer accounts in inline iframes during playtest. Required a probe endpoint (GET /api/me) and a graceful lock screen instead of teasing dead buttons.

4. Gemini rate-limits at demo time

Hit the daily quota mid-recording. Built a SHA-hashed response cache so identical posts (very common during testing) never re-burn quota:

const cacheKey = `ai:${sha256(content)}`
const cached = await redis.get(cacheKey)
if (cached) return JSON.parse(cached)

…then swapped to gemini-2.5-flash-lite for its 4× higher daily allowance.

5. Two mods, one item

Without coordination, two mods can both click "remove" on the same post a second apart. Solved with short-TTL Redis claim locks plus a "pending action" column for high-stakes actions requiring second-mod approval.

Accomplishments that we're proud of

  • 🎯 The Dossier pivot. Building it as connective tissue instead of a sixth tab is the design choice the whole product hinges on.
  • 📈 Adaptive AI threshold. It learns from your team's actual approve/remove decisions instead of asking you to guess a number.
  • 🪶 Zero new triggers for major features. Dossier, Copilot, and Adaptive Threshold all run on data the existing triggers already write. Every new feature got cheaper than the last.
  • 📦 A bundle that fits Devvit's iframe. ~166 KB gzipped for a five-tab React app with two slide-out panels, charts, and an AI chat thread.
  • 🔌 Eleven triggers, one router. All Reddit events fan into a single triggers.ts dispatcher — one place to add, one place to debug.
  • 📬 The Weekly Digest actually fired — with real insight. Devvit's scheduler triggered our weeklyDigest cron on the test sub r/PassiveIncomeHQ and DM'd a markdown breakdown to the head mod:

ModCommand Weekly Digest — r/PassiveIncomeHQ Week ending 5/25/2026 · Total mod actions this week: 39

Per-Moderator Breakdown

  • u/rajkamal2819: 28 actions (72% of team workload)
  • u/modcommand: 9 actions (23% of team workload)
  • u/vineet_2222: 2 actions (5% of team workload)

A 72/23/5 workload split across three mods — exactly the silent imbalance the Workload Wall is built to surface. The head mod sees this in their inbox every Monday, with zero effort. Burnout is no longer something you find out about in an exit interview.

What we learned

  • Integration beats addition. The most valuable feature we shipped (Dossier) added zero new data — it just connected dots that already existed. The hackathon instinct is to keep adding tabs; the better instinct is to keep adding bridges.
  • Trigger-driven state changes everything. Once we accepted that triggers — not the UI — own the source of truth, every UI question got simpler: the dashboard is a view, not a controller.
  • AI is a teammate, not an oracle. Copilot's most useful mode isn't its initial verdict — it's the chat thread where a mod can ask "why did you flag this?" and push back. Confidence labels and showing the heuristics mattered more than raising accuracy.
  • Devvit Redis is a stronger primitive than it looks. Sorted sets + hashes + TTLs cover ~90% of what we'd reach for Postgres for, with zero infra to manage.
  • The demo is the spec. Every time we couldn't demo a feature in under 10 seconds, the feature was wrong — usually too many clicks, too many tabs, too much explaining. Cutting until the demo flowed cut real complexity out of the product.

What's next for ModCommand

Phase 3 — making the system feel alive

  • 📌 Pin-to-Dossier. Mods pin a user → 🔍 badge follows them everywhere across every tab. The system starts watching with you.
  • 📨 Modmail-to-Dossier linkback. Surface "3 modmails in last 30d" inside the Dossier. Modmail stops being a separate inbox and becomes another signal.
  • 🐝 Brigade Watch. Cross-post semantic clustering to detect coordinated raids — the one feature we deliberately deferred until Dossier proved the integration thesis.
  • 🟢 Live presence. See which mods are looking at which items right now via Devvit realtime channels. Kills the last category of accidental double-action.

Longer term

  • Outcome-tracked recommendations. Every Copilot suggestion gets followed by a mod decision. Feed that loop back so the model learns this specific community's norms.
  • Cross-sub mod profile. A mod who runs five subs sees a unified action history and digest, not five separate dashboards.
  • Public moderation transparency. Opt-in per-sub stats page — "this community removed X% of reports, average response time Y." Trust is built on visibility.

The goal isn't to make moderation automatic. It's to make sure the human at the wheel always has the full picture in front of them.

Built With

Share this project:

Updates