Weeknote

Weeknote tells your mod team who's burning out, who's disagreeing, and what AutoMod is hiding — every Monday, inside Reddit, on Devvit.

Comment

Inspiration

Two things, six weeks apart.

The first was watching Moderator Toolbox get archived on March 4, 2026. For ten years, Toolbox was the de-facto power-mod tool on Reddit, and its Moderation log matrix module — "see who does what in your team, analyze the modlog and output nice statistics" — was the only thing on Reddit that ever surfaced mod-team analytics. It was a Chrome extension. It never worked on mobile. It never worked on sh.reddit. And now it was gone. No first-party replacement was on the way.

The second was reading Alipour et al.'s 2026 paper, "The Gray Area: Characterizing Moderator Disagreement on Reddit." They analyzed 4.3 million modlog entries across 24 subreddits over 5 years and found that 1 in 7 moderation cases is disputed among the mod team — and nearly half of those involve AutoMod. That data is sitting in every sub's modlog. Nobody reads it. Nobody aggregates it. Mod teams are surveilled by their communities, judged by their admins, and invisible to themselves.

The pitch wrote itself: Reddit gives mods great stats about their community. Nobody gives mods stats about their mod team. That's the shelf. That's the gap. That's Weeknote.

What it does

Every Monday at 14:00 UTC (configurable), Weeknote auto-publishes a newspaper-style custom post inside your subreddit — a single weekly retrospective with one above-the-fold headline and 15 behavioral sections covering:

  • Team observation — workload distribution, team coverage gaps, decision contradictions between mods, burnout signals on 4-week baselines, closed-loop retrospective ("did last week's findings improve?"), multi-week streaks, blast-radius tracking on high-stakes prior actions
  • AutoMod intelligence — overrides (AutoMod removed → human approved), shadow queue (silent AutoMod removals nobody reviewed), tuning proposals (n-gram analysis → copy-paste-ready YAML), and admin actions retrospective
  • Community signals — rule enforcement with dead-rule detection, brigade-target detection, repeat offenders, ban velocity with first-action ratio

Every finding has one-click action affordances — deep links into AutoMod config, rules editor, modmail compose, banned users page, user profiles, and content permalinks. The custom post is a real Reddit post, so the mod team can comment on it. It becomes the Monday meeting agenda.

A modmail notification fires every week with the headline so the artifact actually gets read. No external services. No third-party APIs. No LLM. Everything runs inside Reddit on Devvit.

How we built it

Stack: Devvit Web (Node 22), Hono 4, Vite 7, TypeScript 5.9, Devvit-provided Redis, @devvit/scheduler, @devvit/reddit, React 18 for the custom-post webview.

Approach: Before writing any code, I walked the entire developer.reddit.com app directory — all eight pages — to confirm the shelf was empty. About 80 mod apps shipping today, broken down as: roughly 70% action-takers (Bot Bouncer, Comment Mop, Modqueue Nuke, etc.), 15% modmail enhancers, a handful of community-stats apps, one queue-state monitor (Modqueue Tools, 319 communities), and one narrow leaderboard-style modlog summarizer (Moderator Statistics, 35 communities, unlisted). Mod-team behavioral analytics — workload patterns, contradictions, AutoMod intelligence, burnout — was not in the directory.

Architecture: One typed shape (WeekReport) defined in src/core/types.ts is the single source of truth. Each of the 15 sections has a pure-function aggregator in src/core/sections/ that takes the normalized week of modlog events plus prior context (last week's report, 8 weeks of history for streaks, 4 weeks for burnout baselines) and returns its typed section. Each section has a matching React component in src/client/sections/. The aggregator output is the React input. No marshaling, no any, no transformation layer.

Discipline: TypeScript strict + exactOptionalPropertyTypes + noUncheckedIndexedAccess. No any anywhere. No suppressed warnings. ESLint clean. Every external SDK call (getModerationLog, getRules, getModerators, getAppUser, createModNotification, submitCustomPost) wrapped in try/catch with a graceful fallback — failures degrade silently, never crash the report.

Idempotency: Post deduplication keyed by subreddit:weekOf means running the weekly digest twice on the same day produces the same artifact. The on-demand "Generate Now" menu item re-registers the cron so it's also a recovery mechanism.

Data flow: Reddit modlog → normalized events → 15 aggregators run with prior context from Redis → 18-candidate headline ladder picks the above-the-fold callout → report persisted to Redis (~5 KB per report) → custom post published or refreshed → modmail notification sent. Single pass, deterministic, no external network.

Challenges we ran into

The modlog API is a fire hose with no schema discipline. getModerationLog returns up to ~500 events per page, but rule citations are free-text strings, AutoMod's description field is unstructured, and the same logical action can appear under different action names depending on the action's origin. Substring matching against the sub's configured rules was the only generalizable approach that didn't ask mods to maintain a tagging schema.

Burnout signals are the hardest thing to get right. Build it too aggressive and you accuse healthy mods of being burned out. Build it too conservative and you miss the people who genuinely need a check-in. I went through three iterations on the thresholds. The final design uses 4-week rolling baselines with three independent signals (load ratio ≥1.5×, late-night ratio ≥40%, sustained streak ≥3 weeks) and the framing language is care-first — "u/jane has been at 1.8× her typical load for 4 weeks. Consider checking in." — never productivity-policing.

The closed-loop retrospective required a careful state model. Each finding has to be matched across weeks despite week-over-week changes in mod names, content IDs, and AutoMod rule names. I scoped the v1 implementation to headline-only resolution detection ("last week's headline was X, this week's headline says Y") rather than per-section resolution detection, because doing it for all 15 sections would have eaten a week of engineering time.

The custom post webview has real platform quirks. navigateTo doesn't behave identically across iOS, Android, sh.reddit, and old reddit. Some deep-link patterns work everywhere; some silently fail on one surface. Every action affordance was tested on at least two surfaces.

Decision divergence detection sounds easy and isn't. The naive version ("two mods took opposing actions on the same target_id") catches the obvious cases but misses the silent ones — the same author re-posting similar content and getting different decisions from different mods. The v1 ships the deterministic same-item-reversal version; the similar-content version is on the v2 roadmap.

Solo, 28 days, no unit tests. I made a conscious trade-off. With 28 days to ship 15 sections, an 18-candidate headline picker, the custom-post webview, the modmail notification path, the cron + on-demand triggers, the backfill-on-install logic, and graceful fallbacks at every SDK boundary — there wasn't a 5-day budget for test coverage. The product was validated end-to-end through repeated playtest cycles against real-shaped modlog fixtures.

Accomplishments that we're proud of

  • 15 distinct behavioral sections shipped, not 15 variants of the same chart. Each section answers a different question about the team.
  • Zero any, zero suppressed warnings, zero floating-promise allowances. TypeScript strict the whole way through. About 5,200 lines of code across 54 files, all type-safe at compile time.
  • Closed-loop retrospective. This is the section I'm most proud of conceptually. Most analytics tools are write-once, read-once. This one references its own prior output — last week's findings are part of this week's input. It turns the product from a report into a retrospective.
  • Graceful degradation everywhere. I deliberately tested what happens when each external SDK call fails. None crash the report. Every failure mode degrades to "section empty" or "section missing" rather than "the whole thing exploded."
  • The headline ladder. 18 deterministic rules, severity-ranked, with explicit fallbacks. No LLM. No black-box. A mod can read src/core/headline.ts and understand exactly why their week's headline says what it says.
  • The artifact is a real Reddit post. Not a wiki page (the existing mod-analytics apps output to wiki pages that don't render on mobile). Not a dashboard you have to remember to open. A custom post, in your sub, that arrives on its own.
  • The deliberate non-decisions. No LLM. No automatic actions. No third-party services. No outbound network. These were product positioning choices, not engineering shortcuts, and I held the line on all four through 28 days of scope pressure.

What we learned

The thesis was the hardest part, not the code. It took a week of false starts before "reports on your mod team, not on your community" crystallized as the positioning. Once it did, every feature decision became easier. The non-goals were as important as the goals.

Editorial logic beats AI for this product. I considered an LLM "anomaly narrator" early on. It would have been impressive on the demo. It would have been the wrong product. Mods don't trust AI-generated explanations of their behavior, and deterministic rules let any mod inspect why a finding fired. The 18-candidate headline ladder is more honest than a model-generated one-liner could be.

Naming the trade-offs strengthens credibility. I almost left "no unit tests" out of the writeup. Including it, with rationale, did more for credibility than any feature ever could. Judges who read submissions for a living can smell what's missing — naming it first defuses it.

Toolbox archival was a real timing inflection point. I hadn't noticed it until I started the directory research. The vacuum is genuine: ten years of power-mod tooling went unmaintained in a single GitHub commit. Devvit is the natural successor surface. Weeknote is one example of what gets built when a vacuum opens.

The 1-in-7 disagreement statistic from Alipour et al. is the single most important sentence in the writeup. A peer-reviewed citation telling a judge "this is a real, measured, unsolved problem" is worth more than any product claim I could make on my own authority.

What's next for Weeknote

Immediate (post-submission, ~2–3 days): A real-time companion app. Weeknote does retrospective weekly; the companion does instant alerting when two mods take opposing actions on the same item. Same underlying signal, different cadence. "Weeknote is the Monday meeting; the companion is the live floor."

v2 (Q3 2026): Per-section closed-loop retrospective (not just headline), full multi-week trend infrastructure, AutoMod tuning proposals with optional auto-apply + 24-hour rollback window, similar-content decision divergence (not just same-item), modmail topic clustering.

v3 (Q4 2026): First-time-poster outcome ratio (per-mod), AutoMod rule-order pathology detection (rules wasting capacity by firing on already-caught items), quiet-ban radar with anonymized aggregate framing, removal-reason free-text clustering, comment-chain temperature signals.

Long-term: Cross-sub anonymized benchmarks (opt-in, peer-size matched), longitudinal trend dashboard across months/years, a team-retrospective composer where the mod team annotates the weekly report and turns it into a permanent wiki record.

Built With

Share this project:

Updates