TriageGuard

TriageGuard turns chaotic mod reports into a prioritized, rule-aware workflow—mods see what to tackle first, why it's urgent, and act safely with one-click tools.

Comment

Inspiration

Moderators on busy subreddits face the same problem every day: the mod queue is a FIFO pile of reports. A scam post, a routine disagreement, and a duplicate link all look the same until you open each one. During the Reddit Mod Tools hackathon, we asked: what if mods had hospital-style triage for their queue—see the worst items first, with a clear why, before taking action?

We were inspired by how crowded the “AI moderation” space is—and deliberately chose not to build another black-box toxicity scorer. Instead we built queue intelligence: fast heuristics, your subreddit’s own wiki rules, and an explainability panel mods can trust.

What it does

TriageGuard is a Devvit mod tool that:

  1. Ingests reported and Automoderator-filtered posts and comments via Reddit triggers.
  2. Scores each item (0–100) using heuristics: report volume, account age, karma, blocked domains, keywords, repeat offenders, and Automod reasons.
  3. Prioritizes items into bands: Critical, High, Routine, and Likely OK.
  4. Explains urgency in a polished dashboard—bulleted “why prioritized,” a matched wiki rule quote, and a non-binding suggested action (e.g. likely remove).
  5. Enables action via mod menu: Approve, Remove (with REMOVE confirmation in audit mode), and Open Dashboard.

Mods open a pinned TriageGuard — Mod Triage custom post (moderator-only) instead of scrolling the native queue blindly. The tool does not replace Reddit’s mod queue UI—it provides a parallel, prioritized work list with deep links.

How we built it

We built on Reddit Devvit (@devvit/public-api):

  • Triggers: PostReport, CommentReport, AutomoderatorFilterPost, AutomoderatorFilterComment, plus AppInstall to seed a dashboard post and wiki cache.
  • Scoring engine: Pure TypeScript in src/config/scoring.ts, unit-tested with Vitest.
  • Storage: Installation-scoped Redis sorted set for open items (tg:open by urgency score).
  • Rules: Wiki page fetch + 24h cache; fallback rule pack if wiki is empty.
  • Optional enrichment: Groq (llama-3.1-8b-instant) for Critical/High items—structured JSON for matchedRule and oneLineWhy, with rate limiting and cache.
  • UI: Devvit Blocks custom post dashboard with dark theme, band filters, and expandable explain panel (our primary UX investment).

Architecture, deployment, and UI specs live in docs/ARCHITECTURE.md, docs/DEPLOYMENT.md, and docs/UI_UX.md.

Challenges we ran into

  1. No native mod queue API — Devvit cannot reorder Reddit’s queue. We reframed the product as a triage command center with deep links, not “we sort your queue.”
  2. Blocks UI limits — Inline approve/remove on the dashboard isn’t reliable; we routed destructive actions to post/comment menus + confirmation forms.
  3. Crowded hackathon category — Many teams ship “AI mod queue” tools. We differentiated with wiki-aligned rules, heuristic explainability without LLM, and audit mode by default.
  4. Scope vs. deadline — We cut modmail alerts, raid signals, and duplicate clustering to ship a polished explain panel and killer demo path.
  5. useAsync + JSON — Dashboard state had to serialize triage items for Devvit’s Blocks hooks.

Accomplishments that we're proud of

  • A screenshot-ready explain panel that shows band, confidence/score, signals, and quoted wiki rules—designed for Moderator’s Choice judges.
  • Zero-config heuristics that work on install; LLM is enrichment, not a dependency.
  • Audit-first design: auditMode defaults on; removals require typing REMOVE.
  • Full SRS-driven build with judging traceability, tests, and documentation.
  • End-to-end flow: report → Critical band → explain → menu remove → cleared from triage list.

What we learned

  • Moderators trust explainability more than model sophistication—a clear “why prioritized” beats another risk score.
  • Rule-aware moderation (wiki citations) is a stronger story than generic “AI moderation” on Devvit.
  • Devvit’s trigger + Redis + custom post pattern is enough for a credible hackathon MVP if scope is ruthlessly protected.
  • Positioning matters: queue intelligence for moderators resonates; “LLM moderation” does not.

What's next for TriageGuard

  • Devvit Web + React client for split-pane list + sticky explain sidebar.
  • Modmail alert on new Critical items (stretch cut from MVP).
  • URL duplicate clustering for spam waves.
  • App Directory launch and Reddit Developer Funds eligibility as installs grow.
  • Optional: per-sub LLM opt-in vs. developer-funded keys for v1.

Built With

Share this project:

Updates