modsteward

Mod notes, scheduled actions, and probation lifecycles. Automated.

Comment

What inspired ModSteward

Every active subreddit has the same three operational problems, and they all come from the same root cause: mods are humans, and humans forget things.

A mod stickies a megathread for an event. Three days later the event is over and the megathread is still pinned, looking stale. A mod locks a heated thread to let tensions cool, intending to unlock in a few hours; two weeks later it's still locked. A user gets put on a temporary AutoMod filter ("two weeks should be enough"), and the filter stays in place for six months because nobody remembered to remove it.

The same problem applies to context. When Mod A removes a user's post for breaking Rule 4, that decision lives in Mod A's head. Three weeks later when the user posts again and Mod B handles it, Mod B has nothing to go on. Every mod is effectively starting from scratch every time.

We noticed these patterns across the subreddits we moderate and across r/ModSupport, where the same complaints surface repeatedly:

  • Reddit provides no native scheduled mod actions
  • There's no probation lifecycle management
  • There's no automatic note-writing on removals

Existing Devvit apps fill other gaps. FlairAssistant handles flair logic, modmail-userinfo surfaces user history at appeal time, Bot Bouncer detects bot patterns. None of them touch these three workflows.

ModSteward exists to handle the pieces of moderation work that mods currently keep in their heads, so they can focus on judgment calls instead of administration.

What we learned

Lesson 1: Focused beats feature-dense

We came into this hackathon assuming "more features = better submission" and started by scoping ModSteward to include a unified dashboard alongside the three core modules. After watching feedback patterns in the Devvit Discord, where mod-judges like SampleOfNone, fsv, and PitchforkAssistant repeatedly favored small, focused tools, we made what felt like a hard cut: deferring the dashboard to v1.1 and shipping only the three background modules.

That cut turned out to be the most important decision in the project. The modules became sharper, the codebase became easier to verify, and the pitch became something we could explain in one paragraph. SampleOfNone put it best in another submission's thread: "Give me 20 apps that all do a small thing well instead of a mega all-in-one thing."

Lesson 2: Destructive operations require paranoia

AutoMod wiki edits can break a subreddit's entire filtering infrastructure if done carelessly. The sentinel block approach took multiple iterations to get right:

  • Clearly delimited start and end markers
  • Byte-identical preservation of surrounding content
  • Single-writer discipline to prevent race conditions
  • Modmail failover on any edit error, rather than silent failure

Verifying it against realistic multi-rule wiki configurations was the single highest-risk test in the project.

Lesson 3: Honest limitations score better than glossed-over ones

Documenting limitations openly felt counterintuitive at first. Things like the BOT_BAN label semantic stretch, the citedRule field that Devvit doesn't yet expose, the cold-start gap for pre-install content, and the AutoMod race condition that's advisory rather than enforced. It seemed like advertising weaknesses. But every conversation we had with mod-judges confirmed that honest acknowledgment of limitations scores better than glossing over them.

How we built it

ModSteward is built on Devvit Web (@devvit/web 0.12.24) with TypeScript, fully server-side. All persistence uses app-scoped Redis. There are no external services and no telemetry.

The three core modules

Removal Scribe

When a mod removes content, Removal Scribe fires on the ModAction event and fetches the body immediately, before Reddit replaces it with [removed]. It then writes a native mod note on the author, with a label that escalates based on their removal history in a rolling 30-day window:

  • First offense: SPAM_WARNING
  • Second offense: ABUSE_WARNING
  • Third and beyond: BOT_BAN

Each write uses exponential backoff with three retries and an idempotency key, so duplicate events never produce duplicate notes.

Mod Action Scheduler

Mods can queue a future action on any post via the context menu. v1 ships with:

  • Lock at 6h or 24h
  • Unsticky at 6h or 24h
  • Remove at 24h

When the scheduled job fires, ModSteward checks the post's current state first. If the action is already done (a mod handled it manually), it posts a skip note instead of executing again. If it executes, a confirmation goes to mod discussions via modmail.

Probation Bot

Places a user under a time-limited AutoMod filter with one click. ModSteward writes a sentinel block into the AutoMod wiki page, bounded by explicit start and end markers. The regeneration function reads the page, locates the block, and rewrites only the content between the markers; everything outside is preserved byte-for-byte. When probation expires, a scheduled job removes the user from the block, writes a closing mod note, and confirms via modmail. If the wiki edit fails, it retries 3 times at 10-second intervals before sending a modmail alert asking the mod to update manually.

Shared features that build on Removal Scribe

Three additional features share the same body archive that Removal Scribe writes:

  • Dynamic label escalation reads recent removal counts from the Redis index to determine which label to apply
  • Escalation alerts post to mod discussions when a user crosses three removals in 30 days, with a 48-hour deduplication key so the same user doesn't trigger repeated alerts
  • Appeal Context posts an internal note to mod discussions when a user with prior removals contacts the sub via modmail, surfacing their most recent removed content before any mod has responded

Settings

Two toggles ship in v1:

  • removalScribeEnabled — the master on/off switch
  • includeBodyInNote — for privacy-sensitive subreddits where removed content shouldn't be stored verbatim

Both default to on. The tool works from the moment of install with no configuration required.

Challenges we ran into

AutoMod wiki editing

This was the highest-risk part of the project. Subreddit AutoMod configs are arbitrary YAML, sometimes with hundreds of rules, and touching the wrong character breaks filtering for the entire subreddit. The sentinel-block approach meant ModSteward only ever touches content between its own markers and treats everything outside as opaque text. Any edit failure falls over to a modmail notification rather than retrying silently. Verifying byte-identical preservation against realistic multi-rule fixtures was the most careful testing we did in the project.

Spec drift between code, tests, and documentation

As the project evolved, the mod note format went through three revisions. At one point the production code, the tests, and the spec all disagreed on whether a field should include the u/ prefix. The tests caught it on a pre-publish check, but only because we held the line on a tests-must-pass discipline. The takeaway: format decisions need to be locked in one place (a shared formatter helper, in our case) so the three artifacts can't drift independently.

The dashboard deferral

ModSteward originally included a unified dashboard as a custom Devvit post. Building it surfaced a dual-storage problem (the mod menu wrote to one Redis schema, the dashboard wrote to another) and 15 test mocks that didn't match the production sorted-set calls. Rather than ship a half-tested dashboard, we deferred it to v1.1 and kept the code in the repo as documented deferred work. This was the hardest call we made. It felt like throwing away significant work, but it made v1 noticeably stronger.

Live runtime verification

Unit tests verify code behavior, but the Devvit runtime has quirks no test simulates: how modmail confirmations render on mobile, how mod notes appear when hovered, whether scheduled jobs fire on time against real Reddit infrastructure. The live test on r/modsteward_dev was where the project actually became production-ready. The most important moment was confirming that AutoMod sentinel-block regeneration preserves a realistic 4-rule configuration byte-for-byte.

The label semantic stretch

Reddit provides exactly five mod note labels and none of them cleanly fit "user with three or more removals in 30 days." BOT_BAN's literal meaning is "this is a bot account we banned," which is a stretch. We had two options: accept the stretch and document it openly, or drop labels entirely and lose visual escalation in the mod notes panel. We went with BOT_BAN and documented the limitation explicitly in the README, betting that mod-judges would respect the honesty more than they'd penalize the imperfect fit.

What's next

The three core modules cover the most common pain points, but there's a clear v1.1 roadmap based on what we couldn't fit into v1 and what came up in feedback from other mods.

The dashboard

The biggest deferred item is the unified dashboard: a custom Devvit post that surfaces all active probations and scheduled actions in one place, with controls to extend, cancel, or run them immediately. The backend for this is already written and tested. What's left is wiring the v1 mod menu storage schema to the v2 dashboard schema and shipping the frontend. We're treating this as the first thing out the door after the hackathon.

Settings the mods actually asked for

A few configuration gaps came up repeatedly when we showed ModSteward to other mods:

  • A configurable escalation threshold (the current "3 removals in 30 days" is hardcoded)
  • Per-subreddit probation duration defaults so mods don't have to pick a duration every time
  • An option to suppress modmail confirmations for high-volume subs where the extra messages become noise

Cited rule in mod notes

Right now the mod note shows what was removed and when, but not which rule it violated. Devvit's ModAction event doesn't currently expose the cited rule field, so this is blocked on a platform change rather than something we can build today. When that field becomes available, it's the single highest-value addition to the Removal Scribe note format.

Broader scheduler actions

v1 ships with lock, unsticky, and remove. The next obvious additions are approve, unlock, and sticky, which cover the "set a post live at a specific time" use case that a few larger subs asked about specifically.

The core philosophy stays the same going forward: one job per module, works at install with no setup, delivers through native Reddit surfaces rather than custom UI wherever possible.

Built With

Share this project:

Updates