ModQA Signals

Communities track content. ModQA tracks moderation quality with weekly QA briefings for Reddit mod teams.

Comment

Inspiration

Moderators already have tools to track content, reports, and queues. But many teams still need a lightweight way to review the quality of their moderation process itself.

ModQA Signals was inspired by a simple question: after a busy week, how can a mod team quickly see where decisions may be drifting, where reports are getting stale, and where a rule may need clearer wording?

What it does

ModQA Signals is a Reddit Devvit mod tool for weekly moderation quality assurance.

It does not replace AutoModerator, classify toxicity, detect spam, or make final moderation decisions. Instead, it surfaces process signals and second-look recommendations so moderators remain in control.

The app generates a weekly QA briefing around three explainable scores:

  • Decision Drift: similar cases with mixed approve, remove, or ignore outcomes.
  • Review Debt: stale reports, unresolved queue items, or repeated revisits.
  • Rule Ambiguity: rules with mixed outcomes, appeals, restores, or ignored reports.

The home screen includes an interactive ModQA Signal Network. It maps raw activity into cases, scoring signals, and second-look recommendations. Moderators can inspect nodes, view tooltips, drag nodes, zoom, and pan to understand where process attention may be needed.

How we built it

ModQA Signals is built as a Devvit mod tool using TypeScript and React.

The app includes a deterministic scoring engine, a responsive moderator-focused UI, and a mock moderation data source so judges can try the product immediately without private subreddit data.

The architecture uses a moderation event source adapter:

  • MockModerationEventSource for demo mode.
  • DevvitModerationEventSource as the integration layer for real moderation data when available.

The scoring logic is separated from the UI so future rule-based signals can be added without rewriting the product.

Challenges we ran into

The biggest challenge was balancing usefulness with safety. Moderation tools can easily feel accusatory or overly automated, so ModQA uses neutral language such as “may deserve a second look” and “consider reviewing.”

Another challenge was making the Signal Network useful without becoming visual noise. We added zooming, panning, draggable nodes, detailed tooltips, and overflow handling so the chart can stay readable as production data grows.

Accomplishments that we're proud of

We are proud that ModQA Signals is concept-complete, installable as a Devvit app, and usable immediately in demo mode.

We also built a scoring core that is deterministic, explainable, and covered by tests. Instead of producing a black-box judgment, each score connects back to visible evidence in the UI.

The interactive Signal Network became a strong product moment: it helps moderators understand how raw activity becomes cases, quality signals, and second-look recommendations.

What we learned

The strongest moderator tools are not always the ones that make decisions automatically. Sometimes the highest-impact tool is one that helps teams reflect, align, and improve their process while keeping humans in control.

We also learned that explainability matters as much as scoring. A score only helps moderators if they can quickly understand where it came from and what they can do next.

What's next for ModQA Signals

Next steps include wiring real Devvit moderation surfaces into the event source adapter, adding subreddit-level scoring settings, deep-linking second-look cards to Reddit mod surfaces, and persisting weekly briefing snapshots for week-over-week trends.

Future versions could also support configurable scoring thresholds, tracked rules, and week-over-week comparisons so moderator teams can see whether their process quality is improving over time.

Built With

Share this project:

Updates