-
idle
-
results
-
expanded
Inspiration
Every moderator who has used AutoModerator knows the anxiety of pushing a new rule live. Will it catch what you intended? Will it nuke legitimate posts? There's no sandbox — the only way to find out is to deploy and watch the damage.
I found the ModSandbox paper (Cornell, CHI 2022) which studied this exact problem. Moderators systematically overestimated how well their rules would perform. The paper called for "audit tools to help moderators tune automated mechanisms." No such tool existed in the Devvit ecosystem, so I built one.
## What it does
AutoMod Lab lets you paste any AutoMod YAML rule into an editor and run it against your subreddit's real post history — before it goes live. The app fetches recent posts and comments, evaluates the rule locally, then compares results against what your mods actually did (approved, removed, marked spam) using the moderation log.
You get back: matched posts with the triggering text highlighted, false positives, false negatives, and precision/recall scores.
## How I built it
Built entirely on Devvit Web (React + TypeScript + Hono). The AutoMod YAML parser and rule evaluator run client-side — no external API calls. Ground truth comes from Reddit's moderation log via the Devvit platform API. Redis caches scan results so repeat runs don't re-fetch the same posts.
Stack: Devvit Web, React 19, Hono, js-yaml, Redis (Devvit KV).
## Challenges
The hardest part was replicating AutoModerator's matching semantics faithfully — includes-word has specific word-boundary rules,
full-exact is case-insensitive by default, regex flags differ from standard JS. I built a test suite against known AutoMod
behavior to catch edge cases.
The Devvit typed API for moderation log entries was incomplete, requiring careful field aliasing to handle real-world data.
Built With
- devvit-web
- hono
- react-19
- redis
- typescript
