The Council

Mods flag borderline content, the team votes privately, and the decision executes itself. No Discord, no unilateral calls, no lost context. Built native on Devvit.

Comment

The Council

What actually breaks moderation teams isn’t spam — spam is easy to handle. It’s the borderline cases: the post that might violate Rule 3, the comment that feels wrong but technically may be allowed. One mod removes it and gets backlash in modmail. Another leaves it up and the subreddit slowly shifts. The team debates it in Discord, reaches no real consensus, and a week later the exact same situation happens again because nothing was ever documented. The action gets logged. The reasoning disappears.

Over time this creates a real problem. New mods join with no context on how the team thinks. Long-tenured mods leave and take their judgment with them. The same borderline situations get handled differently depending on who's online, what mood they're in, and whether they remember the last time something similar came up. Subreddits drift not because the rules changed but because the people applying them changed, and there was never any institutional memory to anchor them.

I've seen this happen. I wanted to fix it. That became The Council.

The idea

What if the decision itself lived inside Reddit? Not a recommendation. Not a flag. Not a Discord thread that gets archived and forgotten. The actual decision — made by the team, recorded permanently, executed automatically, searchable forever.

Every other moderation tool I looked at assumes one mod makes the call. AutoModerator is a single-mod rule engine. Toolbox is a single-mod workflow tool. Even the most sophisticated bots surface information to one mod who then acts alone. The Council is built on a different assumption: the team makes the call, and the system carries it out.

How a vote works

Any moderator clicks Bring to Council on any post or comment. A form appears asking for a brief reason and a vote duration — anywhere from 30 minutes to 24 hours. They fill it in and click Open Council. That's the entire interaction from their side.

Behind that click, several things happen in sequence. The app fetches the target content from the Reddit API and takes a snapshot — title, body excerpt, author, permalink, creation time. It auto-tags the case based on the content and the trigger reason: rule tags matched against a keyword dictionary, media type inferred from the URL domain, content tokens extracted from normalized text. It generates a unique case ID. It creates a private Council post using submitCustomPost() with the default entrypoint, then immediately calls remove() on that post so it disappears from the public feed. Regular users never see it exists. A modmail notification goes to the entire mod team via modMail.createModNotification(), containing the reason and a direct link to the Council post.

Each mod opens the Council post. They see the content that was flagged, the reason the triggering mod gave, the auto-generated tags, and the current vote tally. They vote Keep, Remove, or Warn. Votes can include a reasoning note — up to 500 characters, saved permanently. The tally updates live as mods vote, polling the backend every five seconds.

When the vote closes, the decision runs itself:

  • Keep → reddit.approve() is called on the original post or comment
  • Remove → reddit.remove() is called, and addModNote() attaches a note to the author's account referencing the case
  • Warn → modMail.createConversation() sends a message to the content author with the team's anonymized reasoning notes

No mod has to remember to take the action. No one has to follow up. No one has to check whether it was done. The system handles execution and records the result.

The vote storage problem I didn't expect

The first implementation stored votes directly inside the case object. Read case → append vote → write case. That works fine until two moderators vote at nearly the same time. Both requests read the same state. Both modify it. The last write wins. One vote disappears silently, with no error and no indication anything went wrong.

This is a classic read-modify-write race condition, and it's particularly bad here because the whole point of the system is that every vote counts. Losing a vote isn't just a bug — it's a correctness failure that undermines the legitimacy of the decision.

The fix was moving active votes into a Redis hash — one field per moderator:

council:v1:case-votes:<caseId>
  GolfNo9858   → {"choice":"remove","note":"clear rule 3","timestamp":1716000000000}
  NightShift   → {"choice":"keep","note":"satire exemption applies","timestamp":1716000001000}

hSet updates are atomic and independent per field. Two mods voting simultaneously on different devices cannot overwrite each other — their writes go to different fields in the same hash. The case JSON stores votes: {} while voting is in progress; the hash is the source of truth. When voting closes, markDecided() reads the entire hash, snapshots it into the case JSON, and deletes the hash. The case record is always consistent — either it has no votes (voting in progress) or a complete immutable record (decided). There is no intermediate state.

When does the vote close early?

I didn't want votes sitting open after the result was already obvious. If four mods have voted Remove and zero have voted Keep or Warn, there's no reason to wait for the remaining mods or the timer. The outcome is already certain.

After every vote, the system checks whether the winner can still change. Let $d$ be the leading vote count, $r$ the runner-up, and $n$ the number of human moderators who haven't voted yet. The vote closes immediately when:

$$d > r + n$$

At that point, every remaining moderator could vote for the runner-up and the result would still be unchanged. The condition is a tight bound — it closes at the earliest mathematically valid moment, not a moment sooner.

One thing that broke this in testing: moderator lists include service accounts, AutoModerator, and app bots. Including them inflated $n$ and the condition never held — votes always waited for the timer even when the outcome was obvious. The fix was filtering bot accounts by pattern matching (devvit-*, *-bot, automoderator, reddit) before the calculation runs. Only human moderators count toward $n$.

There are also three other paths that can close a vote: the Devvit scheduler fires at expiresAt, a mod manually clicks Finalize after quorum is met, or maybeFinalizeOnRender() catches a missed scheduler job when the Council post is opened after the timer has already expired. All four paths call the same finalizeCase() function, which is idempotent — if the case is already decided, it returns the existing result without re-executing.

Decision logic and tie-breaking

Once the vote closes, decisionFromTally() determines the outcome. If total votes are below the configured quorum, the result is no-quorum and no action is taken. Otherwise, the plurality winner takes it. Ties are resolved by a configurable tie-breaker setting — the default is extend, which treats a tie as no-quorum and takes no action. Mods can change this to keep or remove in the app settings if they want a different behavior.

The full decision matrix:

Outcome What executes
Remove reddit.remove() + addModNote() on author
Keep reddit.approve()
Warn modMail.createConversation() to author with anonymized reasoning
No quorum No action — logged to Playbook
Cancelled No action — logged to Playbook

All execution is wrapped in try/catch. Failures are recorded in executedActions with success: false and the error message, so mods can see exactly what happened and take manual action if needed.

Auto-tagging

Every case is tagged at creation time with no external APIs. The tagger runs over the combined text of the title, body excerpt, URL, and trigger reason:

  • Type tags: type:post or type:comment
  • Media tags: media:image, media:video, media:link, or media:text — inferred from URL domain matching against known hosts
  • Rule tags: rule:harassment, rule:spam, rule:misinformation, rule:nsfw, rule:brigading, and others — matched against a keyword dictionary
  • Content tokens: top 4 meaningful tokens from normalized text after stopword removal, stored as kw:<token>

These tags serve two purposes. They power Playbook search and filtering. And they're the input to the precedent matching algorithm.

The Playbook

Every decided case is stored in a searchable Playbook — a custom post pinned to the subreddit, visible only to mods. Regular users who navigate to the URL see a placeholder. The Playbook is the team's institutional memory.

Mods can search by keyword across title, body, and author. They can filter by decision type — removed, kept, warned, no quorum. They can filter by tag and sort by recency, oldest first, or most votes. Each case in the list shows the decision badge, the content title, the author, the tags, the vote count, and how long ago it was decided. Clicking a case opens the full detail view: the complete content snapshot, the vote tally, every reasoning note from every mod who voted, the executed actions, and the case metadata.

The Playbook compounds in value over time. A team that has been running The Council for six months has a written record of every borderline call they've ever made, with the reasoning behind each one. A new mod joining that team can read through past cases and understand how the team thinks before they ever have to make a call themselves. That's onboarding that currently doesn't exist anywhere on Reddit.

Precedent matching

The backend already computes similarity scores between cases. When a new case is created, findPrecedents() scores past decided cases using a weighted combination of structural and semantic similarity with a recency decay:

$$\text{score}(c) = 2\,|T_{\text{new}} \cap T_c| + 3\,J(W_{\text{new}},\, W_c) + \frac{1}{1 + \Delta t/30}$$

where $T$ is the tag set, $J$ is Jaccard similarity over normalized content tokens, and $\Delta t$ is the age of case $c$ in days. Tag overlap captures structural similarity — same rule category, same content type. Token similarity captures semantic similarity — similar words in the content. The recency term ensures recent precedents rank higher than old ones with the same score.

kw: tags are excluded from overlap scoring to avoid noise from common words. Only rule:*, media:*, and type:* tags contribute to the structural score.

The scoring runs in the backend. The next version surfaces the top matches directly on the vote page — so before casting a ballot, moderators can see how the team handled comparable situations in the past.

Building it

The project started in Devvit Blocks — the standard component system. That lasted about two weeks before hitting hard limits. No scroll support. Button text invisible in dark mode because appearance="bordered" renders dark text on a dark background with no way to override it. No real search input. No way to build the Playbook UI I wanted.

Three days before the deadline I rewrote the entire frontend to Devvit Web: a Hono server running on @devvit/web/server exposing REST endpoints at /api/* and Devvit hook endpoints at /internal/*, with a React 19 + Tailwind CSS 4 frontend built with Vite. The backend survived the rewrite completely unchanged — storage, voting, finalization, and tagging logic already lived behind a small interface layer.

The bridge between the old backend and the new server is a twelve-line context adapter:

export function makeContext(): any {
  return {
    redis,
    reddit,
    scheduler,
    subredditName: devvitContext.subredditName,
    subredditId:   devvitContext.subredditId,
    postId:        devvitContext.postId,
  };
}

The backend never knew anything changed.

The strangest bug in the whole build: "forUserType": "moderator" in devvit.json silently hides menu items in playtest mode. No error, no warning, the items just don't appear. Spent hours on this. The fix was removing the restriction from config entirely and doing the permission check server-side in the Hono route handler instead.

The second strangest: submitCustomPost doesn't exist on the old @devvit/public-api Reddit client. A one-file type shim casts the client to a compatible interface so the entry parameter passes through without breaking TypeScript.

What's next

The precedent matching is already running. The next version surfaces the top matches on the vote page before a mod casts their ballot — so the decision is informed by what the team has done before, not just the mod's individual memory.

The anonymize voters setting is stored correctly in the data model but the Playbook UI doesn't yet vary based on it. A team that wants fully anonymous voting needs the case detail view to hide voter identities. That's the next UI addition.

The Playbook becomes more valuable the longer a subreddit uses it. That's the compounding effect I built toward — not just a tool for today's decision, but a record that makes every future decision easier.

Built With

Share this project:

Updates