Moderator Decision Memory

AI-powered precedent memory that helps mod teams make consistent borderline calls by surfacing how similar cases were handled before — with transparent reasons and semantic matching.

Comment

Inspiration

Moderators of rule-heavy communities (r/science, r/legaladvice, r/movies) make hundreds of borderline calls daily. New mods have no way to see how the team handled similar cases before — they start from scratch every time, leading to inconsistent moderation and repeated internal debates.

What it does

Moderator Decision Memory gives subreddit mod teams a searchable, retention-aware memory of past moderation decisions. When a borderline item enters the mod queue, the tool surfaces similar prior decisions — showing the outcome, reasoning, and a reusable reply template — so the next moderator can act consistently without starting from scratch.

How we built it

  • Devvit Web 0.13 (React 19 + TypeScript + Vite + Hono server)
  • Devvit Redis for all decision storage (per-installation, subreddit-scoped)
  • OpenAI text-embedding-3-small for semantic precedent matching — so a differently-worded case still surfaces the right precedent while every match shows transparent reasons
  • Hybrid ranking that combines keyword/rule-tag scoring with cosine similarity over embeddings, with graceful fallback to keyword-only if no API key is set
  • onModAction trigger for passive decision capture from real moderation
  • Scheduler with daily cron for retention cleanup
  • Menu actions + Forms for structured decision input on real posts/comments
  • Real mod-queue integration via reddit.getModQueue, reddit.approve, reddit.remove

Challenges we ran into

  • The Devvit Web server must explicitly call createServer + listen (just exporting a Hono app is silently dead — every endpoint returns fetch failed with no error)
  • reCAPTCHA on app creation blocks automated testing (navigator.webdriver)
  • Embedding vectors in Redis require careful serialization alongside the decision JSON

What we learned

  • Semantic matching dramatically improves precedent retrieval — in our integration test, a query about vote manipulation with zero keyword overlap with a brigading precedent went from a 3-way tie (score 4) to correctly surfacing brigading at score 31 (49% semantic similarity)
  • Devvit Web apps need the server bootstrap (createServer + getRequestListener + getServerPort) — this is not documented prominently and cost significant debugging time

What is next

  • Auto-suggested rule tags from post content via LLM
  • Conflict detection when a new decision contradicts past precedents
  • Natural language search over the precedent store
  • Draft explanation templates generated from decision context

Built With

Share this project:

Updates