ModSandbox

ModSandbox lets Reddit mods instantly test and debug AutoMod YAML rules against real posts before deploying, eliminating false positives and saving hours of manual cleanup.

Comment

🧪 ModSandbox: Test Before You Wreck

A developer-grade AutoMod rule testing environment — built natively inside Reddit with Devvit.

💡 Inspiration

Every Reddit moderator knows the sinking feeling of deploying a new AutoMod rule only to discover — hours later — that a tiny syntax error or an overly broad regex pattern has accidentally removed hundreds of legitimate posts.

Moderators are left dealing with flooded modmails, upset community members, and the tedious task of manually restoring content. Before ModSandbox, there was simply no safe, isolated way to test AutoMod rules before making them live.

Our inspiration was simple: build a developer-grade local testing environment directly within Reddit. A safe space where moderators can paste a YAML rule, see exactly which recent posts it matches, and deploy with complete peace of mind.

🛠️ How We Built It

ModSandbox is a native Reddit app engineered on Devvit with a modern full-stack architecture:

  • Frontend: A responsive React 18 client styled with Tailwind CSS, designed to seamlessly adapt to both Reddit's light and dark modes.
  • Backend: A lightweight Hono server wrapper running in Reddit's serverless environment, using tRPC v11 for fully end-to-end type-safe API communication.
  • Core Engine: A custom-built YAML parser and rule matcher written in pure TypeScript. It fetches recent posts via the Reddit API, parses the AutoMod rule schema, and runs regex/contains match logic locally to generate granular visual diffs and highlight matching substrings inline.
  • Storage: Subreddit-specific drafts and test history are persisted safely using Devvit Redis.

📐 Modeling Moderator Time Complexity

To measure our impact, we mathematically modeled the expected moderator time spent per rule deployment, \(E(T)\), before and after using ModSandbox.

Without ModSandbox, the expected time wasted is:

$$E(T){\text{traditional}} = P(\text{Error}) \times \left( T{\text{restore}} \cdot N_{\text{impacted}} + T_{\text{rewrite}} \right) + (1 - P(\text{Error}}) \times T_{\text{verify}}$$

Where:

  • \(P(\text{Error})\) — probability of a rule containing a bug or false positive
  • \(T_{\text{restore}}\) — time to manually restore a single incorrectly filtered post
  • \(N_{\text{impacted}}\) — number of posts falsely flagged before the error is noticed
  • \(T_{\text{rewrite}}\) — time spent correcting the rule

With ModSandbox, \(P(\text{Error})\) approaches zero:

$$\lim_{P(\text{Error}) \to 0} E(T){\text{sandbox}} = T{\text{sandbox_verify}}$$

By reducing \(P(\text{Error}) \approx 0\), we effectively collapse the costly manual restoration term — saving communities hours of administrative overhead per deployment.

⚠️ Challenges We Ran Into

Building a complex, interactive web application within an iframe inside the Reddit environment presented unique engineering challenges:

  1. Developing Without Hot-Reload Devvit's current pipeline requires a manual build and upload step for every change. Waiting ~\(15\) seconds for each compile-upload-register cycle forced us to write highly modular, unit-tested code in our /shared directory before integrating into the React frontend.

  2. Deterministic Regex Engine AutoMod relies on specific regex behaviors. Replicating its parsing quirks — case-insensitive comparisons, edge punctuation handling, list-matching logic — within a sandboxed Node.js runtime required rigorous testing against real-world AutoMod edge cases.

  3. Optimizing Payload Size Running tRPC over iframe postMessage boundaries meant structuring response payloads efficiently. We built a custom text segmenter that sends only highlighted match coordinates to the client, rather than heavy pre-formatted HTML structures.

🎓 What We Learned

  • The Power of Sandboxing: Giving users a visual playground builds immediate trust. Seeing highlighted words in real posts makes complex YAML rules instantly understandable — even for non-technical moderators.
  • Modular Code is King: Decoupling the parser and matching engine from the React UI and tRPC server let us write isolated Vitest test cases, drastically mitigating the lack of fast refresh in the Devvit runtime.
  • Designing for Accessibility: Subreddit moderation happens everywhere. Ensuring our UI loaded fast, looked premium, and fully respected Reddit's dark and light system themes was paramount for a great native user experience.

🔭 What's Next for ModSandbox

ModSandbox is just getting started. Our roadmap includes:

  • [ ] Supporting deeper AutoMod conditions like author_flair and link_flair
  • [ ] Comment-level testing to intercept spam and toxic threads before they propagate
  • [ ] Version control and rule diff history saved to Devvit Redis for seamless mod team collaboration

Built With

Share this project:

Updates