vibe-mod - Say It, Ship Subreddit Rules

Say a subreddit rule in plain English — vibe-mod compiles it into a deterministic, auditable, reversible rule. Zero AI calls per post, free for mods, 30-day undo.

Comment

Inspiration

Most subreddit rules start as a sentence, not YAML. A mod sees the same pattern all week - fresh accounts dropping links, ALL-CAPS titles, short low-effort posts - and knows exactly what should happen. The hard part is turning that judgment into automation that is safe enough to run on a live community.

AutoModerator is powerful, but it asks volunteer mod teams to maintain YAML, regex, and exact syntax. A small mistake can catch the wrong posts, and a new rule can affect a community before anyone has seen what it would do.

The product line became: say it, ship subreddit rules. The safety line became just as important: the model can help write the rule, but it should never be the judge of the community.

We built vibe-mod for the gap between "I know the rule I want" and "I trust this rule enough to turn it on."

What it does

vibe-mod is a Devvit mod tool that turns a moderator's plain-language rule into a validated subreddit rule. A moderator can type something like "send links from accounts under 7 days old to the mod queue," preview what it would catch, save it as a draft, and let it run in shadow mode before it takes real action.

It's a general rule compiler, not a fixed list of checks: mix 34 post, author, time, and report signals x 9 operators x 9 moderation actions into essentially any rule you can describe - account age, karma, post length, links, flair, time-of-day, report counts, and more.

The AI runs only when a moderator clicks Compile, and only on the sentence the moderator typed. If that sentence is ambiguous, vibe-mod asks a clarifying question instead of guessing. After that, the saved rule runs as deterministic TypeScript with $0 per-post AI cost - the model never reads subreddit posts, comments, usernames, or community history.

The moderator experience is:

  • Compose rule: write the rule in normal language.
  • Compile + Preview: vibe-mod validates the compiled rule and shows what it would do before anything happens.
  • Shadow first: new rules log decisions for 24 hours before going live.
  • Reacts to mod actions: rules can fire on your own moderation too - apply the "Spam" flair and the matching rule removes and locks the post automatically.
  • Manage rules: pause, activate, delete, and inspect active or draft rules.
  • Undo this action: every live action keeps a 30-day rollback path.

Subreddit moderators do not need an OpenAI key or billing account. vibe-mod covers AI compilation up to a daily quota, then enforces saved rules without model calls.

How we built it

vibe-mod is a server-only Reddit Devvit app built with TypeScript. Devvit handles the moderator menu actions, form submissions, post/comment/report triggers, scheduled jobs, permissions, and subreddit-scoped Redis storage.

There are two main paths:

  • Compile path: moderator sentence -> OpenAI gpt-5.4-mini -> strict JSON rule -> Zod parse -> action whitelist -> draft rule in Devvit Redis.
  • Runtime path: Devvit trigger -> fact bag -> deterministic evaluator -> shadow log or live action -> audit entry and rollback token.

The rule evaluator is intentionally pure TypeScript. It evaluates a closed set of facts from the Reddit item, author context, and subreddit-scoped state. It does not call OpenAI, perform network I/O, or make stochastic decisions. That is the core guarantee: AI helps author the rule; deterministic code applies it.

We also built the boring parts that make the idea usable: strict schema validation, guarded moderation actions, a subreddit-level dry-run switch, action rate limits, audit entries, rollback tokens, scheduler checks, starter rules, Terms and Privacy docs, and CI.

Challenges we ran into

The hardest decision was saying no to the obvious AI bot. It would have been flashier to call a model on every post, but moderators need consistency, privacy, and predictable cost. A model that reads every submission is expensive and hard to audit. A compiled rule is cheaper, repeatable, and easier to explain.

That choice made the implementation stricter. Model output has to pass a Zod schema, then pass an action whitelist, then be stored as a draft. If the request is ambiguous, vibe-mod asks a clarifying question instead of guessing. If a rule would use guarded actions like ban, mute, permaban, or approve, the moderator has to opt in explicitly.

Devvit also shaped the implementation. The app had to fit Devvit's permission, settings, Redis, scheduler, and server-bundle model. We added a doctor script to check config integrity, declared only one fetch domain, and kept OpenAI key management as a developer-owned encrypted global setting so subreddit moderators do not need to bring keys or billing.

Finally, multi-rule behavior required an honest scope decision. The current app executes every matching rule in order, but it also surfaces potential conflict warnings in the dashboard. That tells moderators where rules may collide without pretending the app can solve every policy question automatically.

Accomplishments that we're proud of

  • vibe-mod is live in the Reddit App Directory.
  • The model is used as a rule compiler, not a content judge.
  • Runtime moderation has $0 per-post AI cost.
  • New rules get preview and shadow mode before live action.
  • Every action has a 30-day undo path.
  • Guarded actions require explicit moderator opt-in.
  • Subreddit moderators do not need their own OpenAI key or billing account.
  • The public repo includes README, architecture notes, developer docs, Terms, Privacy Policy, MIT license, icon, thumbnail, and landing page.
  • The current suite has 236 passing tests, 1 skipped test, CI, Devvit harness checks, acceptance gates, build checks, and server-bundle smoke loading.

What we learned

The safest use of AI in moderation is to narrow its job. A language model is useful for translating moderator intent into structure. It should not be the thing making live moderation decisions on every post.

We also learned that trust features are not polish. Preview, shadow mode, audit logs, rate limits, guarded actions, and undo are what make natural-language moderation usable by real mod teams. Without those controls, "say the rule" would be convenient but risky.

On the platform side, Devvit rewards small, explicit systems. Declaring exact permissions, keeping state in subreddit-scoped Redis, and proving route/config parity made the app easier to review and easier to explain.

What's next for vibe-mod - Say It, Ship Subreddit Rules

Next, we want to add richer rule signals while keeping the deterministic model intact: more account-history facts, richer link/domain patterns, repost indicators, edited-content facts, and better language signals.

We also want to make rule management stronger: conflict warnings during compose, an explicit promotion gate for high-risk conflicts, rule import/export between subreddits, and a dashboard with hit-rate history and false-positive notes.

Longer term, vibe-mod can become a migration path from hand-written AutoModerator configs: read an existing AutoMod rule, translate it into vibe-mod's structured schema, preview the result, and let the moderator decide whether to switch.

Built With

Share this project:

Updates