ModOS

Solo mods spend 20 min/day reading a flat queue. ModOS ranks it by risk, explains every flag in plain English, and lets you act in one tap — 91 hours returned per year.

Comment

ModOS — Full Submission Document

Reddit Mod Tools Hackathon · Best New Mod Tool All submission copy + architecture diagrams in one file.

Inspiration

Meet Priya. She mods r/IndieDevs solo — 47,000 members, one person, no days off.

Every morning she opens Reddit's mod queue: 60+ items, flat list, no ranking, no explanation. Spam posts look identical to genuine questions. A coordinated affiliate campaign sits next to a first-time poster asking for help. Reddit shows everything equally — and she has 20 minutes before her real job starts.

Reddit's mod queue tells you what's waiting. It doesn't tell you what matters.

The sub grows. Two new mods join: Marcus, who handles routine approvals while Priya sleeps in a different timezone, and Aiko, who covers the EU morning rush. Now there's a new problem: they're stepping on each other. Marcus approves something Aiko was reviewing. Priya can't tell if the queue is clear or if something slipped through. There's no shared view, no role separation, no workload visibility.

Then a coordinated spam campaign hits. 8 posts in 12 minutes from the same account. By the time Priya sees the first one, the other 7 are buried in the queue.

ModOS exists because moderation is the only job where the tools assume you work alone, have infinite time, and never get attacked.

What it does

ModOS is a Devvit-native intelligence layer that sits on top of the Reddit mod queue. Install it in your subreddit, open it from mod tools, and you get five tabs that turn reactive chaos into a proactive system.

Triage — Smart, Ranked Queue

Every post and comment gets scored the moment it's submitted using 7 deterministic signals. The queue auto-sorts into High Risk / Medium / Normal / Noise buckets — mods always know where to look first.

Every score is fully explained: plain-English chips like New account (2 days old), Same domain 4× this week, Author burst. No black boxes. A mod can read, agree with, or challenge every flag.

Inline Approve / Remove. Bulk-approve all Normal/Noise items in one click. Queue health alert: red banner fires if a HIGH-risk item has been waiting 30+ minutes — nothing slips through while a mod is offline.

Shield — Coordinated Attack Detection

Every 3 minutes, ModOS scans for author bursts: accounts posting ≥4 times in 15 minutes. When detected, a cluster card appears with a single ⚡ Nuke campaign button that removes every item in the burst as spam in one action. A 10-second response to what used to be a 10-minute cleanup.

With AI enabled, each cluster card shows an AI campaign summary: "8 posts promoting a Telegram crypto investment channel, all using identical shortened links." — so mods know exactly what they're nuking before they tap.

Team — Role-Based Mod Coordination

Assign roles to each mod:

  • Senior Mod — HIGH + MEDIUM risk (dangerous items, first line of defence)
  • Triage Mod — MEDIUM + NORMAL (judgment calls)
  • Janitor Mod — NORMAL + NOISE (bulk approvals, routine backlog)

Each mod's Triage automatically filters to their assigned bucket range. No overlap, no duplicate work. Priya, Marcus, and Aiko each see only what they're responsible for.

The admin dashboard shows every mod's personal stats. A workload balance warning fires when one mod is carrying 2× the team average — so the lead mod can rebalance before anyone burns out.

Settings — Per-Sub Customization

Every subreddit is different. ModOS is fully tunable:

Sensitivity presets (Low / Balanced / High) set baseline thresholds in one tap.

Signal tuning — enable/disable any of the 6 built-in signals per-sub. Override individual weights. A sub about new developers might want to disable "New account" entirely. A crypto sub might crank "Repeat domain" to 80.

Custom keyword signals — teach ModOS what spam looks like in your community. Add a keyword (e.g. t.me/, udemy, a competitor's domain), set a weight and chip label. Every future post containing it fires that signal. Each keyword shows a hit counter (fired: 7×) so you can see whether your teaching is working.

AI content analysis (Claude) — opt-in AI review layer. Enable it in Settings, paste your Anthropic API key once, and every HIGH or MEDIUM item gets reviewed by Claude Haiku in the background (every 5 minutes). Each card gets an AI verdict chip: AI: likely spam, AI: looks legitimate, or AI: uncertain — with a one-sentence reason from Claude alongside the deterministic signals. (Fully implemented; requires Devvit HTTP permission activation — outbound HTTP to api.anthropic.com is blocked in Devvit's current sandbox environment.)

AI keyword suggestions — describe your spam in plain English ("Telegram links promoting crypto courses") and Claude generates 3-5 specific keyword rules with weights and chip labels. Review the suggestions and add them with one click. (Same platform constraint applies.)

All customizations survive preset switches — keywords and weight overrides are preserved when switching sensitivity levels.

Insights — Prove the Value

  • Items handled, high-risk surfaced, estimated time saved
  • Personal stats per mod (your contribution, not just the team's)
  • Signal accuracy based on "Good catch / Wrong call" feedback
  • Accuracy recommendations — if a signal is wrong >30% of the time (with ≥5 feedback samples), ModOS tells you to disable or retune it
  • Audit log — every approve/remove with timestamp, mod name, bucket, signal chips. Full accountability trail.

Auto-Escalation DMs

If a HIGH-risk item sits unreviewed for 30+ minutes, ModOS sends a Reddit private message to the Senior Mod. The team lead gets paged even when the app isn't open. No repeat spam — each item is marked alerted.

How we built it

Built on Reddit's Developer Platform with one optional external integration.

  • Devvit Web (React 19 + Tailwind CSS 4 + Vite) — UI inside a Reddit post
  • Hono — lightweight server for triggers, scheduler, and all REST API routes
  • Devvit Redis — all state: items, assessments, clusters, config, mod roles, keyword stats, audit log, counters
  • Devvit Triggers (PostSubmit, CommentSubmit, PostReport, CommentReport, ModAction, AppInstall)
  • Devvit Scheduler — Shield scan + escalation scan (every 3 min) + AI review scan (every 5 min)
  • Claude API (opt-in, implemented) — Anthropic's Claude Haiku reviews flagged items for spam. API key stored per-community in Redis. Scheduler processes up to 10 items per run. Verdict + reason stored in RiskAssessment.aiReview; displayed as a purple chip on each item card. permissions.http.domains declared in devvit.json. Live calls blocked by Devvit's server sandbox in current deployment; outbound HTTP to api.anthropic.com is not permitted in the playtest/sandbox environment.

Scoring pipeline: Trigger fires → runSignals() evaluates 6 built-in signals + custom keyword rules (7th signal type) → score() sums weighted outputs → bucket assignment → RiskAssessment written to Redis. UI reads from cache instantly.

Role system: mod:roles hash in Redis stores username → ModRole. /api/triage reads calling mod's identity via reddit.getCurrentUser(), maps to role, filters items to that role's allowed buckets.

Per-sub customization: CommunityConfig stores keywords[], signalWeights, disabledSignals. applyPreset() typed as Omit<CommunityConfig, 'keywords'|'signalWeights'|'disabledSignals'> — preset switches only touch threshold fields, preserving all community tuning.

Keyword hit tracking: Each CUSTOM_KEYWORD FiredSignal carries a ruleId. On every trigger, matching rule IDs increment cnt:kw:{ruleId} counters. Settings tab reads these to show live hit counts.

Audit log: Every /api/act call writes an AuditEntry (mod, action, item title, bucket, signal chips) to a Redis list, capped at 200 entries. Insights tab shows the last 20.

Challenges we ran into

No redis.keys. Devvit Redis doesn't support key scanning. Every collection needs an explicit hash index (hSet/hDel/hKeys). This constraint pushed us toward cleaner architecture — O(1) index lookups instead of scan-everything.

Trigger payload quirks. PostV2.createdAt is epoch milliseconds already — not seconds. author.createdAt doesn't exist on the trigger payload; getting account age requires a separate getUserByUsername() call per submit.

Explainability over accuracy. Every signal must produce a human-readable chip and a plain-English clause that stays accurate across edge cases (unknown account age, missing domain, zero-karma new accounts). A signal that fires correctly but can't explain itself is worse than useless for a mod auditing a decision.

Role filtering without locks. Multiple mods viewing the same queue simultaneously need different filtered views without items being "claimed." Roles are filters, not locks. Every item stays in the active index; each mod's view is filtered server-side by their role's allowed buckets.

Preset + customization compatibility. Presets change thresholds cleanly. But communities spend time configuring keywords and signal weights. Typed PRESETS as Omit<CommunityConfig, 'keywords'|'signalWeights'|'disabledSignals'> — preset application only touches threshold fields, preserving all tuning.

Accomplishments we're proud of

  • Zero false "black box" scores. Every score is traceable to a named signal with a weight, chip, and reason clause. Mods can always answer "why is this flagged?"
  • Nuke campaign. One button removes a coordinated spam burst. 8-item cleanup in under 10 seconds.
  • Custom keyword DNA. Subreddits teach ModOS to recognize their own spam patterns — specific domains, competitor mentions, community-specific slang — making the tool genuinely personalized.
  • Role coordination that works. Three mods, three queue slices, no overlap. Workload balance warning catches burnout before it happens.
  • Signal accuracy feedback loop. Mods rate catches as good or wrong. ModOS computes per-signal accuracy and proactively recommends disabling underperforming signals. The tool improves with use.
  • Full audit trail. Every mod action logged with timestamp, mod name, bucket, and signal chips. Production-grade accountability.
  • AI layer fully implemented (platform-blocked). The Claude integration — verdict chips, cluster summaries, keyword suggestions — is fully coded, UI-wired, and unit-tested (19 tests). Live calls are blocked by Devvit's server sandbox which does not allow outbound HTTP in the current environment. permissions.http.domains: ["api.anthropic.com"] is declared in devvit.json; the feature is ready to activate when Devvit enables the HTTP permission for this app.

What we learned

Reddit's trigger system is fast but write-only — you can't do multi-item cross-referencing inside a trigger. This forced a clean separation: triggers score one item at a time and write atomically; the scheduler handles cross-item analysis asynchronously. That constraint produced a more resilient architecture than we'd have built otherwise.

The hardest UI problem in moderation tools isn't showing information — it's hiding it. The noise bucket collapse, role-filtered triage view, signal tuning panel — all about giving mods exactly what they need for their job, nothing more.

Devvit's server sandbox blocks all outbound HTTP — global fetch routes through a gRPC proxy that denies unlisted domains even after declaring permissions.http in devvit.json. Discovering this during live testing meant the Claude integration, though fully implemented and unit-tested, couldn't be demonstrated live. This is a platform constraint we're flagging to the Devvit team.

What's next for ModOS

  • Modmail escalation — auto-escalation to subreddit modmail for full team visibility (instead of single DM)
  • Rule builder — drag-and-drop signal combinator: "if NEW_ACCOUNT + REPEAT_DOMAIN → always HIGH"
  • Confidence-threshold auto-actions — when AI + deterministic signals both exceed a threshold, auto-remove with mod review window
  • AI removal reason drafting — when removing a HIGH confidence item, Claude drafts the removal reason message for the mod to review and send
  • Weekly intelligence brief — scheduled AI summary: top spam patterns, keyword performance, threat trends
  • MCP server — a companion MCP server would let mods query their ModOS queue directly from Claude chat — we're exploring this as a post-launch extension once Devvit supports persistent webhook endpoints
  • Cross-community signal sharing — same domain spamming 40 subs simultaneously gets flagged for all participating mods as a network. (Concept — not functional in v1; requires cross-community Redis infrastructure)

Built with

Devvit · React 19 · TypeScript · Tailwind CSS 4 · Vite · Hono · Devvit Redis · Claude API (Anthropic)

Technical Architecture

System Overview

                        REDDIT PLATFORM
                               |
            PostSubmit / CommentSubmit / Report / ModAction
                               |
                               v
+------------------+    +--------------------+    +----------------------+
|  Devvit Triggers |    | Devvit Scheduler   |    |   Devvit Redis       |
|                  |    |                    |    |                      |
|  PostSubmit      |    |  shield-scan       |    |  item:{id}           |
|  CommentSubmit   |    |  every 3 min       |    |  assess:{id}         |
|  PostReport      |    |  -> burst clusters |    |  active:items  (Hash)|
|  CommentReport   |    |                    |    |  active:clusters (H) |
|  ModAction       |    |  escalation-scan   |    |  cluster:{id}        |
|  AppInstall      |    |  every 3 min       |    |  win:author:*  (ZSet)|
+------------------+    |  -> DM senior mods |    |  win:domain:*  (ZSet)|
         |              |                    |    |  win:texthash: (ZSet)|
         |              |  ai-review-scan    |    |  cfg, mod:roles      |
         |              |  every 5 min       |    |  cnt:*, feedback:*   |
         |              |  -> Claude Haiku   |    |  audit:log    (List) |
         | score + write|  -> update assess  |    |  cnt:kw:*            |
         |              +--------------------+    |  alerted:items (Hash)|
         |                       |                |  ai:pending    (Hash)|
         +-----------+-----------+                +----------+-----------+
                     v                                       |
                     |                                       |
                     +-------- reads / writes ---------------+
                     |
                     v
+----------------------------------------------------------------------+
|                          Hono Server                                 |
|                                                                      |
|  /internal/triggers/*   score + store to Redis                      |
|  /internal/scheduler/*  burst detection + escalation DMs            |
|  /internal/menu/*       create post / seed demo data                |
|                                                                      |
|  /api/triage            ranked queue, filtered by mod role          |
|  /api/act               approve / remove + write audit entry        |
|  /api/insights          stats + personal dashboard + audit log      |
|  /api/clusters/*        burst clusters + nuke campaign              |
|  /api/mod-roles         team role assignment (GET + POST)           |
|  /api/config            sensitivity preset + re-score all active    |
|  /api/keywords/*        custom keyword CRUD + hit counters          |
|  /api/signals/*         per-signal weight override + toggle         |
|  /api/feedback          record good-catch / wrong-call              |
|  /api/audit             last 20 mod actions                         |
+----------------------------------------------------------------------+
                     |
                fetch /api/*
                     |
                     v
+----------------------------------------------------------------------+
|                    ModOS React UI  (Devvit Web Post)                 |
|                                                                      |
|   [Triage]   [Shield]   [Insights]   [Settings]   [Team]           |
|                                                                      |
|   Triage   : ranked buckets, role filter, queue health alert        |
|   Shield   : burst cluster cards, nuke campaign button             |
|   Insights : stats grid, personal stats, audit log, recs           |
|   Settings : presets, signal tuning, custom keywords + hit counts  |
|   Team     : role assignment, workload balance, per-mod dashboard  |
+----------------------------------------------------------------------+

Directory Structure

modos23/
├── devvit.json                     <- manifest: triggers, scheduler, menu, entrypoints
├── vite.config.ts                  <- build: client -> dist/client, server -> dist/server/index.cjs
│
├── src/
│   ├── shared/
│   │   ├── types.ts                <- ALL types: RawItem, RiskAssessment, CommunityConfig,
│   │   │                              AuditEntry, KeywordRule, ModRole, Recommendation...
│   │   └── api.ts                  <- every API request/response shape
│   │
│   ├── server/
│   │   ├── index.ts                <- Hono entry: mounts /api + /internal
│   │   ├── redis.ts                <- all Redis key helpers + typed wrappers (no magic strings)
│   │   ├── windows.ts              <- ZSet rolling-window: push, prune, count, getItems
│   │   ├── score.ts                <- scoreItem(): signals -> weights -> bucket + sentence
│   │   ├── actions.ts              <- reddit.approve() / reddit.remove() with TID handling
│   │   ├── config.ts               <- PRESETS + applyPreset() (set config + re-score all active)
│   │   ├── insights.ts             <- getInsights(): counters -> InsightsSnapshot + recommendations
│   │   ├── ai.ts                   <- analyzeContent() + suggestKeywords() via Claude Haiku API
│   │   ├── queue.ts                <- getQueue(): raw Reddit mod queue fetch (AppInstall backfill)
│   │   ├── seed.ts                 <- runSeed(): 25 mock items for demo mode
│   │   │
│   │   ├── signals/
│   │   │   ├── index.ts            <- runSignals(): disabled check, weight overrides, custom kw
│   │   │   ├── newAccount.ts       <- NEW_ACCOUNT      weight 30
│   │   │   ├── lowTrust.ts         <- LOW_TRUST        weight 25
│   │   │   ├── highReports.ts      <- HIGH_REPORTS     weight 40
│   │   │   ├── repeatedDomain.ts   <- REPEATED_DOMAIN  weight 35
│   │   │   ├── repeatedText.ts     <- REPEATED_TEXT    weight 40
│   │   │   ├── authorBurst.ts      <- AUTHOR_BURST     weight 50
│   │   │   └── customKeyword.ts    <- CUSTOM_KEYWORD   weight configurable (10-60)
│   │   │
│   │   └── routes/
│   │       ├── api.ts              <- all /api/* REST endpoints
│   │       ├── triggers.ts         <- all /internal/triggers/*
│   │       ├── scheduler.ts        <- shield-scan + escalation-scan
│   │       ├── menu.ts             <- /internal/menu/* (create post, load demo)
│   │       └── forms.ts            <- /internal/form/*
│   │
│   └── client/
│       ├── game.tsx                <- app shell: tab state, toast stack, action handlers
│       ├── Triage.tsx              <- bucket groups, role banner, queue health alert, bulk approve
│       ├── Shield.tsx              <- cluster cards, nuke campaign button
│       ├── Insights.tsx            <- stats, personal dashboard, audit log, recommendations
│       ├── Settings.tsx            <- presets, signal tuning, custom keywords + hit counters
│       ├── Team.tsx                <- role assignment, workload balance warning, per-mod stats
│       ├── splash.tsx              <- install landing page
│       ├── components/
│       │   └── ItemCard.tsx        <- full card: chips, sentence, approve/remove, feedback
│       └── hooks/
│           ├── useTriage.ts        <- fetches /api/triage, exposes role + todayStats
│           └── useClusters.ts      <- fetches /api/clusters, dismiss()

End-to-End Flow: Post Submitted -> Mod Acts

USER SUBMITS POST
      |
      | Devvit onPostSubmit fires
      v
1.  Parse payload (post, author)
2.  reddit.getUserByUsername(author.name)  -> authorCreatedAt, karma
3.  Build RawItem { id, kind, authorName, authorCreatedAt, authorKarma,
                    title, body, domain, numReports, createdAt, permalink }
      |
      v
4.  getConfig()  -> CommunityConfig (preset thresholds + custom keywords)

5.  Push + prune rolling windows (parallel):
    pushWindow('author',   authorName, id, createdAt)
    pruneWindow('author',  authorName, cutoff)        cutoff = now - windowMin
    pushWindow('domain',   domain,     id, createdAt)  <- link post only
    pushWindow('texthash', hash,       id, createdAt)  <- if text present

6.  Count window sizes (parallel):
    authorCount = zCard('win:author:{name}')
    domainCount = zCard('win:domain:{domain}')
    textCount   = zCard('win:texthash:{hash}')
      |
      v
7.  runSignals(item, cfg, { authorCount, domainCount, textCount })

    SYNC signals (item fields only):
      NEW_ACCOUNT    -> authorCreatedAt < cfg.newAccountDays * 86400000
      LOW_TRUST      -> authorKarma < cfg.karmaFloor
      HIGH_REPORTS   -> numReports >= cfg.reportFloor

    WINDOW signals (need WindowContext):
      REPEATED_DOMAIN -> domainCount >= 3
      REPEATED_TEXT   -> textCount >= 2
      AUTHOR_BURST    -> authorCount >= cfg.burstFloor

    CUSTOM_KEYWORD signals:
      for each rule in cfg.keywords[]:
        if title+body contains rule.keyword (case-insensitive):
          fire { id:'CUSTOM_KEYWORD', weight:rule.weight, chip:rule.chip, ruleId:rule.id }

    Weight overrides:  if cfg.signalWeights[signal.id] -> override weight
    Disabled signals:  if cfg.disabledSignals includes signal.id -> skip

8.  score = sum of weights of all fired signals

9.  bucket:
      score >= highCutoff         -> HIGH
      score >= highCutoff * 0.5   -> MEDIUM
      score >= 10                 -> NORMAL
      score <  10                 -> NOISE

10. sentence = "Flagged because {clause1}, {clause2}, and {clause3}."
11. suggestedAction = review_now | likely_spam | needs_judgment | probably_safe
      |
      v
WRITE TO REDIS (parallel):
    set('item:{id}',    JSON(RawItem))
    set('assess:{id}',  JSON(RiskAssessment))
    hSet('active:items', { [id]: '1' })
    if bucket==='high' -> incrBy('cnt:highRisk', 1)
    for each fired signal -> incrBy('cnt:reason:{signalId}', 1)
    for CUSTOM_KEYWORD signals with ruleId:
      incrBy('cnt:kw:{ruleId}', 1)
      set('cnt:kw:{ruleId}:last', now)
      |
      v               (item now in Redis queue)
      |
      +-------------------+----------------------------+
      |                   |                            |
      v                   v                            v
MOD OPENS MODOS    SHIELD SCAN (3 min)     ESCALATION SCAN (3 min)
      |                   |                            |
GET /api/triage    group items by author    find HIGH items > 30 min old
  - reddit.getCurrentUser()                check alerted:items hash
  - getModRole(modName)                    DM Senior Mods via reddit.sendPrivateMessage()
  - filter by ROLE_BUCKETS[role]           mark items in alerted:items hash
  - fetch item + assess pairs
  - sort by score DESC
  - return { items, modName, role, todayStats }
      |
      v
React renders Triage tab:
  - HIGH cards at top
  - each card: chips + sentence + feedback buttons
  - inline Approve / Remove
  - queue health alert if HIGH item > 30 min old
      |
      v
MOD TAPS "REMOVE"
  1. dismissItem(id)                 <- instant local state update
  2. POST /api/act { id, kind, action:'remove' }
       - reddit.remove(T1/T3 TID)    <- actual Reddit API call
       - hDel('active:items', [id])  <- remove from queue index
       - incrBy('cnt:handled', 1)
       - recordModAction(modName, wasHighRisk)  <- per-mod counter
       - appendAudit({ ts, modName, action, itemId, title, bucket, chips })
  3. toast "Removed"

Shield: Burst Detection Flow

Every 3 minutes: POST /internal/scheduler/shield-scan
       |
       +- hKeys('active:items')          -> all active IDs
       +- getItem() for each             -> items[]
       +- group by authorName            -> Map<author, itemIds[]>
       |
       +- for each author with >= burstFloor active items:
       |    zCard('win:author:{name}')   -> count in rolling window
       |    if count >= burstFloor:
       |      getWindowItems('author', name)  -> all itemIds in window
       |      mark author BURSTY
       |
       +- upsert Cluster for each bursty author:
       |    id: 'burst:{authorName}'     <- deterministic, safe to upsert
       |    label: 'u/foo: 4 posts in 15 min'
       |    hSet('active:clusters', { ['burst:{author}']: '1' })
       |
       +- remove stale clusters (burst subsided + not dismissed):
            hDel('active:clusters', [id])
            del('cluster:{id}')

MOD TAPS "NUKE CAMPAIGN":
  POST /api/clusters/nuke { id }
    - listClusters() -> find cluster by id
    - for each itemId in cluster.itemIds:
        getItem(id) -> RawItem
        removeItem(id, kind, isSpam=true)   <- spam-flagged removal
        recordModAction(modName, true)       <- counts as high-risk handled
    - removeCluster(id)
    - return { status:'ok', removed: N }
  toast: "Campaign nuked — 8 items removed as spam"

NOTE: AUTHOR_BURST signal fires INLINE on the 4th post (step 7 above)
      before the scanner runs. Scanner is the grouping + display layer.

Escalation DM Flow

Every 3 minutes: POST /internal/scheduler/escalation-scan
       |
       +- listActive()                 -> all active item IDs
       +- for each ID:
       |    getAssessment(id)          -> bucket
       |    isAlerted(id)              -> already DMed?
       |    getItem(id)                -> createdAt
       |    if bucket==='high' AND now - createdAt > 30min AND NOT alerted:
       |      staleHighIds.push(id)
       |
       +- if staleHighIds.length === 0 -> return (nothing to do)
       |
       +- getModRoles()                -> Record<modName, ModRole>
       +- seniorMods = entries where role === 'senior'
       +- if seniorMods.length === 0  -> return (no one to DM)
       |
       +- reddit.sendPrivateMessage({
       |    to: seniorMod,
       |    subject: 'ModOS: Stale HIGH-risk items need review',
       |    text: '3 HIGH-risk items waiting 30+ min in mod queue'
       |  })
       |
       +- markAlerted(itemId) for each item  <- hSet('alerted:items', { [id]: '1' })
          prevents duplicate DMs on next scan run

Role-Based Queue Filtering

GET /api/triage
       |
       +- reddit.getCurrentUser()     -> modName (or 'unknown')
       +- getModRole(modName)         -> ModRole from mod:roles hash
       |
       |  ROLE_BUCKETS:
       |    senior  -> ['high', 'medium']
       |    triage  -> ['medium', 'normal']
       |    janitor -> ['normal', 'noise']
       |    all     -> ['high', 'medium', 'normal', 'noise']
       |
       +- listActive()                -> all item IDs (shared pool)
       +- for each ID: getItem + getAssessment
       +- filter: item.assessment.bucket IN ROLE_BUCKETS[role]
       +- sort by score DESC
       +- return { items, modName, role, todayStats }

Result: three mods open ModOS simultaneously.
  Priya (senior)  sees: 4 HIGH + 3 MEDIUM items
  Marcus (janitor) sees: 12 NORMAL + 8 NOISE items
  Aiko (triage)   sees: 3 MEDIUM + 12 NORMAL items
  No overlap. No duplicate work. Same Redis pool, different filtered views.

WORKLOAD BALANCE WARNING (client-side):
  if any mod.handled > avg(allMods.handled) * 2:
    show amber warning banner in Team tab:
    "u/{name} is handling 3x the team average. Consider reassigning role."

Redis Key Schema

ITEMS                               INDEXES (hash, field=id, value='1')
------------------------------      ------------------------------------
item:{id}        String/JSON        active:items     Hash
assess:{id}      String/JSON        active:clusters  Hash
                                    known:mods       Hash
CLUSTERS                            alerted:items    Hash
------------------------------
cluster:{id}     String/JSON        ROLLING WINDOWS (ZSet, score=ts ms)
                                    ------------------------------------
CONFIG + ROLES                      win:author:{name}
------------------------------      win:domain:{domain}
cfg              String/JSON        win:texthash:{hash}
mod:roles        String/JSON

COUNTERS                            FEEDBACK
------------------------------      ------------------------------------
cnt:highRisk     int                feedback:{signalId}:good   int
cnt:handled      int                feedback:{signalId}:bad    int
cnt:clusters     int
cnt:reason:{id}  int                KEYWORD HIT COUNTERS
cnt:handled:{mod} int               ------------------------------------
cnt:highRisk:{mod} int              cnt:kw:{ruleId}            int
                                    cnt:kw:{ruleId}:last       string (ts)

AUDIT LOG
------------------------------
audit:log        List (max 200 entries, RPUSH + LTRIM)
Key Type Written by Read by
item:{id} String PostSubmit trigger GET /api/triage
assess:{id} String PostSubmit, PostReport GET /api/triage
active:items Hash addActive / removeActive listActive -> hKeys
active:clusters Hash addCluster / removeCluster listClusters -> hKeys
cluster:{id} String Scheduler shield-scan GET /api/clusters
win:author:{name} ZSet Every submit trigger Score pipeline
mod:roles String POST /api/mod-roles GET /api/triage
cnt:handled:{mod} String POST /api/act GET /api/mod-roles
audit:log List POST /api/act GET /api/audit
cnt:kw:{ruleId} String PostSubmit (if keyword fires) GET /api/keywords/stats
alerted:items Hash escalation-scan escalation-scan

Why hashes not sets: Devvit Redis has no SADD/SREM/SMEMBERS. Hash with constant value '1' per field is the equivalent.

Why ZSets for windows: score = createdAt (ms) lets us prune with zRemRangeByScore(key, 0, cutoff) and count with zCard. No manual TTL management.

Signal Pipeline

RawItem + CommunityConfig + WindowContext
         |
         v
runSignals()
         |
         +-- check cfg.disabledSignals -> skip disabled signals
         |
         +-- SYNC signals (pure item data)
         |
         |   NEW_ACCOUNT        weight 30 (overridable)
         |   fires: authorCreatedAt > 0
         |          AND now - authorCreatedAt < newAccountDays * 86400000
         |   chip:   "New account"
         |   clause: "the account is only {N} days old"
         |
         |   LOW_TRUST          weight 25 (overridable)
         |   fires: authorKarma > 0 AND authorKarma < karmaFloor
         |   chip:   "Low karma"
         |   clause: "the author has only {N} karma"
         |
         |   HIGH_REPORTS       weight 40 (overridable)
         |   fires: numReports >= reportFloor
         |   chip:   "{N} reports"
         |   clause: "received {N} community reports"
         |
         +-- WINDOW signals (need WindowContext)
         |
         |   REPEATED_DOMAIN    weight 35 (overridable)
         |   fires: item.domain AND domainCount >= 3
         |   chip:   "Repeat domain"
         |   clause: "links to a domain seen {N} times recently"
         |
         |   REPEATED_TEXT      weight 40 (overridable)
         |   fires: textCount >= 2
         |   text normalized: lowercase + collapse whitespace + djb2 hash
         |   chip:   "Duplicate text"
         |   clause: "uses text identical to {N} other recent posts"
         |
         |   AUTHOR_BURST       weight 50 (overridable)
         |   fires: authorCount >= burstFloor
         |   chip:   "Author burst"
         |   clause: "the author has posted {N} times recently"
         |
         +-- CUSTOM_KEYWORD signals (per-sub rules from CommunityConfig)
             for each rule in cfg.keywords[]:
               if (title + body).toLowerCase().includes(rule.keyword.toLowerCase()):
                 fire { id:'CUSTOM_KEYWORD', weight:rule.weight,
                        chip:rule.chip, ruleId:rule.id }
                 incr cnt:kw:{rule.id}  (async, in trigger handler)
         |
         v
         apply cfg.signalWeights overrides to each fired signal
         |
         v
FiredSignal[] -> scoreItem() -> RiskAssessment

Scoring Model

score = sum of weights of all fired signals

Examples (balanced preset, highCutoff = 60):

  LOW_TRUST(25)                                           = 25  -> NORMAL  (>= 10)
  NEW_ACCOUNT(30)                                         = 30  -> MEDIUM  (>= 30 = highCutoff * 0.5)
  NEW_ACCOUNT(30) + LOW_TRUST(25)                         = 55  -> MEDIUM  (< 60)
  NEW_ACCOUNT(30) + HIGH_REPORTS(40)                      = 70  -> HIGH    (>= 60)
  HIGH_REPORTS(40) + AUTHOR_BURST(50)                     = 90  -> HIGH
  NEW_ACCOUNT(30) + HIGH_REPORTS(40) + AUTHOR_BURST(50)   = 120 -> HIGH
  custom keyword(35) + NEW_ACCOUNT(30)                    = 65  -> HIGH

Bucket thresholds (highCutoff = 60):

  0 ---- 10 --------- 30 ----------- 60 --------------------->
  NOISE  |   NORMAL   |    MEDIUM    |         HIGH
  < 10   |   >= 10    |    >= 30     |         >= 60
                         medium threshold = highCutoff * 0.5

Preset Comparison

Preset Account age Karma floor Reports High cutoff Window Burst
Low < 7 days < 10 >= 5 80 15 min 6 posts
Balanced < 30 days < 50 >= 3 60 15 min 4 posts
High < 90 days < 100 >= 1 40 30 min 2 posts

All presets preserve: keywords[], signalWeights, disabledSignals

API Reference

Method Path Description
GET /api/triage Active items, filtered by mod role, sorted by score
GET /api/clusters Active burst clusters
GET /api/insights Stats + personal dashboard + audit log
GET /api/config Current community config
GET /api/mod-roles All role assignments + per-mod stats
GET /api/keywords/stats Hit counters per keyword rule
GET /api/audit Last 20 mod actions
POST /api/act Approve or remove (calls Reddit API + writes audit entry)
POST /api/config Apply sensitivity preset, re-score all active items
POST /api/feedback Record good-catch / wrong-call on an item
POST /api/clusters/dismiss Remove cluster from active set
POST /api/clusters/nuke Mass-remove all cluster items as spam
POST /api/mod-roles Assign role to a mod
POST /api/keywords/add Add custom keyword rule
POST /api/keywords/remove Remove keyword rule by ID
POST /api/keywords/ai-suggest Generate keyword rules from natural language description
POST /api/signals/weight Override signal weight (or null to reset)
POST /api/signals/toggle Enable or disable a signal
POST /api/ai/config Set aiEnabled flag + store Anthropic API key

Trigger Reference

Trigger Action
onAppInstall Create pinned post + backfill up to 25 existing queue items
onPostSubmit Full processItem(): enrich from Reddit API, score, write to Redis
onCommentSubmit Same as PostSubmit, comment-type RawItem
onPostReport Re-score existing item with updated numReports
onCommentReport Same as PostReport for comments
onModAction Auto-remove from active index on terminal actions

Terminal mod actions that sync ModOS queue:

removelink   approvelink   spamlink
removecomment   approvecomment   spamcomment

Platform Constraints & Solutions

Constraint Solution
No redis.keys or redis.scan Hash index per collection: hSet/hDel/hKeys
No sAdd/sRem/sMembers Hash with constant value '1' per field
ZSet member shape is { score, member } All window code uses object shape
PostV2.createdAt is already epoch ms No x1000 multiplication anywhere
author.createdAt not on trigger payload getUserByUsername() call on every submit
reddit.approve/remove needs T3/T1 TID toTid(id, kind) wraps both
No showToast in @devvit/web/client React state-based toast stack in game.tsx
Triggers must be cheap + write-only Score one item, write Redis, return. No cross-item work.
Scheduler jobs must be idempotent Cluster IDs are burst:{authorName} — safe to upsert
Role filtering without item locking Roles are server-side filters, not locks on items

Type System at a Glance

// Item flowing through the system
RawItem -> scoreItem() -> RiskAssessment -> ScoredItem (merged for client)

// Signal evaluation
(RawItem + CommunityConfig + WindowContext) -> FiredSignal[]
FiredSignal = { id: SignalId, weight, chip, clause, ruleId? }
                                                       ^-- only for CUSTOM_KEYWORD

// Buckets
'high' | 'medium' | 'normal' | 'noise'

// Signal IDs (all deterministic, no ML)
'NEW_ACCOUNT' | 'LOW_TRUST' | 'HIGH_REPORTS'
'REPEATED_TEXT' | 'REPEATED_DOMAIN' | 'AUTHOR_BURST' | 'CUSTOM_KEYWORD'

// Mod roles
'senior' | 'triage' | 'janitor' | 'all'

// Audit entry (every mod action logged)
AuditEntry = { ts, modName, action, itemId, title?, bucket, signalChips[] }

// Community config (fully customizable per sub)
CommunityConfig = {
  preset, newAccountDays, karmaFloor, reportFloor,
  highCutoff, windowMin, burstFloor,        // threshold fields (preset-controlled)
  keywords: KeywordRule[],                  // custom keyword signals
  signalWeights: Partial<Record<SignalId>>, // per-signal weight overrides
  disabledSignals: SignalId[],              // signals to skip entirely
}

Test Coverage

All deterministic logic is unit-tested with Vitest. Tests run with npm test (no mocking, no network calls).

 Test Files  12 passed (12)
      Tests  113 passed (113)
   Duration  ~500ms
File Tests What it covers
signals/newAccount.test.ts 7 Unknown age skip, threshold boundary, day-label accuracy, custom threshold
signals/lowTrust.test.ts 6 At/above/below karma floor, zero-karma skip
signals/highReports.test.ts 6 Singular/plural chip, report floor boundary, custom floor
signals/repeatedDomain.test.ts 5 No domain, count <3 skip, count ≥3 fire
signals/repeatedText.test.ts 4 Count <2 skip, count ≥2 fire
signals/authorBurst.test.ts 5 burstFloor boundary, author name in clause
signals/customKeyword.test.ts 8 Case-insensitive, title+body match, ruleId in output, multi-keyword stacking
score.test.ts 24 All 4 buckets, all suggestedAction branches, sentence construction, weight overrides, disabled signals, preset thresholds, output structure
score.boundary.test.ts 14 Exact threshold edges (scores 9/10/29/30/59/60), all 7 signals simultaneous (score=255), preset comparison, keyword stacking
windows.test.ts 9 textHash: determinism, case normalization, whitespace/tab/newline collapse
ai.test.ts 19 parseVerdictResponse + parseKeywordSuggestions: all verdicts, malformed input, cap at 5, prose wrapping
redis.keys.test.ts 10 All 25 Redis key helpers produce correct strings; no two static keys collide

What's not unit-tested: Redis wrappers (require mocking Devvit SDK), route handlers, React components. These are covered by live playtest on the test subreddit r/modos23_dev.

Built for the Reddit Mod Tools Hackathon · Best New Mod Tool No required external services. Every flag has a reason. Every action is one tap.

Built With

  • devvit-redis
  • devvit-web
  • hono
  • react-19
  • tailwind-css-4
  • typescript
  • vite
  • vitest
Share this project:

Updates