ReputiBot

Inspiration

Reddit moderators are flying blind. The #1 complaint on r/ModSupport (343 upvotes, 327 comments) is that Reddit now lets users hide their post history, making it nearly impossible to identify bot accounts, ban evaders, and trolls. Combined with the wave of AI-generated spam that overwhelms AutoModerator, mods desperately need a tool that builds its own behavioral intelligence per-subreddit.

ReputiBot was born from this pain point: what if every subreddit could automatically build trust profiles for its users, and surface AI-powered risk assessments with a single click?

What it does

ReputiBot passively scores every user's trustworthiness in a subreddit based on three dimensions:

Tenure — how long they've been active in this specific sub
Activity — volume of posts and comments
Reliability — ratio of approved vs removed content (learns from mod decisions)

When a mod needs to evaluate someone, they right-click → "ReputiBot: Check trust score" and get:

A 0-100 trust score with breakdown
An AI behavioral analysis from GPT-5-nano (detects spam, bot patterns, toxicity)
Everything saved as a private Mod Note — visible only to mods, persistent, native to Reddit

It also supports auto-filtering high-risk users to the mod queue, weekly digest reports, and a dashboard post.

How I built it

Devvit Web (0.12) with Hono server framework — the new route-based architecture
Redis for per-user profile storage and sorted sets for score rankings
OpenAI GPT-5-nano with structured output (JSON schema) to guarantee concise responses under 150 characters
Event triggers — PostCreate, CommentCreate, ModAction, PostReport, CommentReport all feed the scoring engine
Reddit API — mod notes, user lookups, post/comment fetching for AI context
Rate limiting, score staleness detection, configurable weights/thresholds

Challenges I faced

Devvit forms can't display read-only data — there's no modal/popup for showing information. Had to iterate through toasts, forms, and comments before landing on mod notes as the perfect native UX.
GPT-5-nano is a reasoning model — it spends tokens "thinking" internally before producing output. With 150 max_completion_tokens, it used ALL of them for reasoning and returned empty. Had to bump to 4000 and use structured output to control response length.
30-second execution limit — combining Reddit API calls + external AI calls within one menu action is tight. Solved by minimizing content sent to AI and using the fastest model available.
No secret settings in new devvit.json schema — isSecret: true caused validation errors. Worked around it with build-time env variable injection via Vite define + global app settings via CLI.

What I learned

The new Devvit Web architecture is powerful but under-documented — the route-based Hono approach is a big improvement over the old singleton API
Mod notes are the ideal output mechanism for mod tools — private, persistent, native
Reasoning models need very different token budgets than traditional models
Redis sorted sets are perfect for "who are the worst N users" queries