Shill Spot

Inspiration

Over the last few years, the way people look for honest opinions has shifted. Review websites and blogs felt increasingly gamed and SEO-farmed, so people started appending "reddit" to their searches to get reviews from what felt like real humans. At the same time, AI overviews and chatbots became the default way to look things up, and much of what they summarize is scraped straight from Reddit threads.

Both of those trends direct a significant stream of trust towards Reddit, making it a target. Brands have every incentive to quietly seed and spike their products into the exact subreddits people go to for recommendations. An experienced user might notice an account that only ever shows up to praise one brand, but a casual user searching for "best X" usually can't. Neither can an AI summarizer; it'll happily repeat a planted recommendation, strip away any context that surrounds it, and present it as consensus.

The result is a slow erosion of what made Reddit valuable in the first place. If enough recommendation threads are astroturfed, the reviews stop being trustworthy, and Reddit loses its status as a forum people can rely on. I built Shill Spot to give moderators a way to catch this kind of covert promotion before it hollows out their community's credibility.

What it does

Shill Spot watches new comments and posts in a subreddit, scores them against a set of behavioral and content signals, and reports anything suspicious to the native mod queue with a plain English explanation. It never removes anything on its own; every flag is a recommendation that a moderator makes the final call on.

It looks for several independent signals:

• Repeat brand mentions — If the same user keeps bringing up the same watched brand across multiple comments and posts within a rolling window, that pattern gets flagged. This is the thing Reddit's own spam filter cannot see structurally, because it evaluates each comment in isolation.

• Blacklisted brands — for products a community considers actively harmful (say, a pet shampoo known to hurt animals), a single mention is enough to flag. A comment warning people about the product is left alone, while one promoting it gets flagged.

• Affiliate links — 17 built-in URL patterns (Amazon Associates tags, Skimlinks, ShareASale, RewardStyle, link shorteners, generic ?ref=/?aff= params). Reddit already blocks the most obvious ones, so this acts as a corroborating signal rather than the main signal.

• Account profile — new accounts and low-karma accounts, the classic throwaway-shill profile.

• Username–brand match — when an account's username contains a brand name that also appears in the body.

Each signal carries a configurable weight, and content is only reported when the total clears a threshold:

$$\text{score} = \sum_{s\,\in\,\text{fired signals}} w_s \cdot m_s \qquad\text{report} \iff \text{score} \ge \text{threshold}$$

where w_s is the mod-configured weight for signal s and m_s is a context multiplier for example, an affiliate link inside a post is weighted 1.5 times the same link in a comment, because a post is a more deliberate act.

Everything is configurable per subreddit through Reddit's native settings page: the watched and blacklisted brand lists, the score threshold, every signal weight, which signals are even active, a trusted user allowlist, and a cooldown that prevents the bot from re-reporting a user for a while after a mod acts on them. There's also a "dry-run" mod menu action that runs any hypothetical comment through the full pipeline and shows the verdict without reporting anything handy for tuning the weights to a community's taste before going live.

How we built it

It's a Devvit Web app written in TypeScript (strict mode). Devvit Web models everything as HTTP: Reddit calls webhook endpoints on a small Hono server when events happen (onCommentSubmit, onPostSubmit, onModAction), and the app talks back through Reddit's APIs.

The codebase is split so each file has one job:

• patterns.ts / detection.ts — pure detection functions (scanForAffiliateLinks, checkRepeatBrand, checkBlacklistedBrand, classifyTone, plus the account-age, karma, and username checks). No I/O, which made them trivial to unit-test.
• scoring.ts — turns a list of fired signals into a score and two reason strings: a short one for the mod-queue report and a full breakdown for the audit trail.
• storage.ts — thin wrappers over Devvit Redis. Each user is one JSON record in a users hash, holding their flag history and a capped, time-pruned list of brand mentions.
• config.ts — reads the moderator's settings and falls back to sane defaults field by field.
• routes/triggers.ts — on each comment or post, load config and the user record, run the signal pipeline, score it, and report if it clears the threshold.

The detection logic is covered by 96 unit tests in vitest, which run independently of Devvit, so they're fast. Live testing happened in a private test subreddit via devvit playtest, with several throwaway accounts playing the roles of shills and ordinary users.

Challenges we ran into

• reddit.report() silently rejects long reasons. Reddit caps a report reason at 100 characters and returns a gRPC TOO_LONG error otherwise. My first reason strings were detailed multi-signal breakdowns, well over that, so every single report was failing. Because it failed deep in an async call, nothing appeared in the mod queue, and nothing obvious appeared in the logs. The fix was to send a summary on the wire (Shill Spot 8.5/5.0: repeat_brand,new_account) and keep the full breakdown in the user's audit record.

• Reddit already solves the easy version of the problem. I started out building an affiliate-link scanner, then discovered while testing that Reddit's own spam filter blocks most obvious affiliate domains before my trigger ever sees them. The valuable, unsolved problem isn't matching URLs, it's the behavior of the same account quietly pushing the same brand over time. That pivot made repeat-brand detection the centerpiece, demoting affiliate matching to a supporting role.

• Sentiment of a comment was an issue. A naive "flag any mention of a banned product" rule flags the people warning their community about it, which is exactly backward. I built a keyword-based sentiment gate, then immediately hit its weakness: "this isn't bad" contains the word "bad" and reads as negative. I added negation handling phrases like "isn't bad" or "wouldn't recommend" to flip the tally to cover the common cases.

Accomplishments that we're proud of

• Every flag explains itself. There's no black box. The report names which signals fired, their weights, and the resulting score, so a moderator can make an informed call in seconds.
• *The sentiment gate *, including the awkward negation cases, so the tool protects the community members who are doing the right thing instead of punishing them.
• It's fully tunable per community thresholds, weights, brand lists, even the sentiment vocabulary, because what counts as suspicious in r/AskHistorians is normal in r/deals.

What we learned

This was my first substantial TypeScript project and my first time building on Devvit, so a lot of it was learning the platform from scratch, how triggers, the Redis plugin, the settings API, and the Reddit client fit together. The bigger lesson was about scoping: I learned to keep asking "does Reddit already do this?" and to aim the tool at the part of the problem that's actually unsolved. I also came away with a concrete feel for the limits of keyword-based sentiment analysis. It's good enough to be useful as a gate, but it's clearly a stepping stone to something smarter.

What's next for Shill Spot

• Cross-account coordination detection catching the case where several new accounts converge on the same brand in a short window, the signature of an organized campaign rather than a lone shill.
• Score decay over time, so an account that cleans up its behavior isn't penalized forever.
• Smarter sentiment replacing the keyword gate with a proper model once it can run within Devvit's constraints.
• Cross-subreddit intelligence sharing (with privacy safeguards), so a shill caught in one community can raise the priors in another.

Built With

• devvit
• hono
• node.js
• redis
• typescript