ShadowMod

• 

  01-mod-tools-menu

• 

  02-observer-record-form

• 

  03-observer-observation-confirmed

• 

  04-mod-tools-review-queue

• 

  05-reviewer-review-queue-modal

• 

  06-reviewer-role-guard-toast

• 

  07-reviewer-record-form

• 

  08-reviewer-review-confirmed

• 

  09-comparison-report-modmail

• 

  10-feed-after-real-actions

• 

  11-settings-configure-reviewers

• 

  12-observer-accuracy-stats

Inspiration

I scraped and sentiment-analysed recent posts on r/ModSupport using VADER and found a 53.2% negativity score on threads about new-mod mistakes, the highest of any topic cluster measured. The pattern is consistent: a new mod takes an action that does not fit community norms, the community notices, and then the experienced mods spend hours in modmail doing damage control.

The root cause is structural. New mods learn on live content, with real consequences, and no feedback loop. Reddit's Training Queue validates the problem and is a solid foundation. It covers approve/remove decisions and delivers immediate per-post feedback. What it does not cover is the full decision vocabulary (flair, ban, escalate), reasoning capture, or a blind independent Review by an experienced mod for comparison over time. ShadowMod picks up where the Training Queue stops.

What it does

ShadowMod adds a structured Observer/Reviewer workflow to any subreddit's mod queue, as a Devvit-native install.

When an Observer (the mod-in-training) encounters a post, they select "Record observation" from the post menu and submit their intended action plus reasoning. The action is not executed.

A Reviewer (an experienced mod the subreddit has designated) later opens the same post, selects "Record review", and submits their own call, blind to what the Observer recorded. No anchoring.

When any mod or AutoModerator takes the real action on the post, ShadowMod fires and schedules a report. The Observer receives a Comparison report via modmail: their Observation, the Reviewer's Review, and the final Outcome, with both reasonings side by side. Over time, the Observer can check their stats: total Observations, agreement rate, accuracy trend.

How I built it

I used AI-assisted development throughout, consistent with how I build software professionally for enterprise clients. The architecture decisions, data model, and design tradeoffs are mine.

Everything runs on Devvit primitives. Post-level and subreddit-level menu actions handle the Observer and Reviewer forms and the Review queue. Redis stores Observations, Reviews, and per-user stats, namespaced per subreddit install. The onModAction trigger fires when the real post action lands and schedules the generate-report job. Report delivery goes to the Observer via modmail, with a mod note fallback when modmail is unavailable.

No external dependencies. One-click install. Reviewer assignment is done through a settings menu action, stored in config:reviewers in Redis, and checked server-side on every request.

Challenges

Enforcing blind Review was the central design problem. The Reviewer must not see the Observer's call before recording their own. This is solved structurally: the Review form fires a separate menu action on the post, the Observation is never surfaced to the Reviewer UI, and the Comparison report is only generated and delivered after both records exist and the real action has landed.

Async report delivery required careful timing. The onModAction trigger fires on the real action, but at that point the Review may not yet exist (the Reviewer works on their own schedule). The system queues reports and waits until both sides are recorded before generating the comparison.

Role-gating without native Devvit roles meant building my own. Observer vs Reviewer is determined by membership in a Redis-stored set, checked on every relevant action. The settings menu lets subreddit admins manage that list without touching Devvit config files.

What's next

v1 ships the core loop: Observation, blind Review, async Comparison report, and Observer stats. The direction for v2 will be shaped by real usage. Once communities start running Observations, the interesting questions become: what decision patterns show up across subreddits, where do Observers and Reviewers consistently disagree, and what does that say about how moderation norms are communicated inside mod teams. I want those insights to come from the communities using the tool, not assumptions made upfront.

Built With

• devvit
• onmodaction-trigger
• reddit-modmail-api
• redis
• typescript