AUTOTRAIN

Advanced Threat Intelligence & Automated Enforcement for Reddit Moderators

Comment

AUTOTRAIN - DevPost Submission

Advanced Threat Intelligence & Automated Enforcement for Reddit Moderators

🛡️ Built for the Reddit Mod Tools Hackathon 2026

🎬 Inspiration

As Reddit communities grow, moderators face an impossible challenge: how do you protect your community from bad actors while giving good-faith users a fair chance?

The inspiration for AUTOTRAIN came from three critical pain points we observed across Reddit:

  1. Mod Burnout - Large subreddits receive hundreds of posts daily. Manually researching each user's history is exhausting and unsustainable.

  2. Inconsistent Enforcement - Without context, different mods make different decisions. One mod might ban immediately while another issues a warning for the same violation.

  3. Lack of Intelligence - Moderators have no real-time visibility into user risk factors. Is this a 2-day-old account? Do they have negative karma? Have they been warned before? Mods have to manually check each time.

We asked ourselves: What if moderators had instant threat intelligence on every user, combined with a fair, graduated enforcement system that gives users a chance to improve?

That's AUTOTRAIN - a tool that brings automated risk profiling, graduated strikes, and behavioral analytics to Reddit moderation, all while keeping humans in control of the final decisions.

🎯 What It Does

AUTOTRAIN is a comprehensive moderation app that provides Reddit moderators with six core capabilities:

1. 🔍 Real-Time Risk Profiling

Every user who posts or comments is automatically scored 0-100 based on:

  • Account age (new accounts = higher risk)
  • Karma distribution (low or negative karma = higher risk)
  • Username patterns (bot-like names = higher risk)
  • Existing strikes in your subreddit
  • Watchlist status (manually flagged users)
  • Posting velocity (spam detection)

Example: A 2-day-old account with 5 karma and a username like "HappyDog1234" posting promotional content would score 95/100 (HIGH RISK).

2. ⚠️ Graduated Strike System

Right-click any post or comment → "Issue Strike"

  • Strike 1: Warning message sent via modmail
  • Strike 2: Final warning message
  • Strike 3: Automatic ban (configurable)

Each strike is logged with the rule violated, mod notes, and timestamp. Users receive clear explanations of what they did wrong and how to improve.

3. 🚨 Anomaly Detection

Background scheduler runs every 5 minutes to detect:

  • Spam waves (5+ high-risk events in 30 minutes)
  • Posting velocity spikes (10+ posts in 1 hour)
  • Coordinated attacks

When detected, AUTOTRAIN automatically creates a mod-only alert post with details of the suspicious activity.

4. 👁️ User Watchlist

Right-click any post or comment → "Add to Watchlist"

  • Elevates monitoring for suspicious users
  • Adds 15 points to their risk score
  • All future posts/comments flagged immediately
  • Easy toggle on/off

5. 🔍 Instant Risk Profile Viewer

Right-click any post or comment → "View Risk Profile"

  • Shows risk score, account age, karma, strikes, flags
  • Instant visibility into user threat level
  • No need to manually check user profiles

6. 📊 Interactive Mod Dashboard

Create via subreddit menu → "Create AUTOTRAIN Dashboard"

  • Custom post showing recent high-risk alerts
  • Strike history across your community
  • Watchlist users
  • Refresh button for real-time updates

🏗️ How We Built It

Technology Stack

Component Technology
Platform Reddit Devvit (Deno-based serverless)
Language TypeScript (strict mode)
Storage Redis KV (provided by Devvit)
Runtime Reddit's infrastructure (no external dependencies)
UI Devvit Blocks (React-like component system)
API Reddit API via Devvit SDK

Architecture

AUTOTRAIN is built as a modular, event-driven application:

User Activity (Posts/Comments)
    ↓
Event Triggers Fire (PostSubmit, CommentSubmit)
    ↓
Risk Scoring Engine (Calculate 0-100 score)
    ↓
Redis Storage (Cache profiles, strikes, alerts)
    ↓
Threshold Checks (Flag high-risk users)
    ↓
Background Scheduler (Detect anomalies every 5 min)
    ↓
Mod Dashboard (Display intelligence)

For detailed technical architecture, see ARCHITECTURE.md

Code Structure

src/
├── main.ts          # Entry point & app registration (50 lines)
├── settings.ts      # Mod-configurable settings (80 lines)
├── risk.ts          # Risk scoring engine (250 lines)
├── triggers.ts      # Event handlers (150 lines)
├── menu.ts          # Context menu actions (300 lines)
├── scheduler.ts     # Background anomaly detection (120 lines)
└── dashboard.ts     # Custom post UI (200 lines)

Total: ~1,150 lines of production TypeScript

Key Design Decisions

  1. Caching Strategy - User profiles cached for 1 hour to reduce Reddit API calls by 90%
  2. Graduated Enforcement - Three-strike system gives users a fair chance to improve
  3. Human-in-the-Loop - All enforcement actions require mod approval (no auto-removal)
  4. Zero Configuration - Works out-of-the-box with sensible defaults
  5. Subreddit Isolation - Each community's data is completely isolated in Redis

🚧 Challenges We Ran Into

1. Balancing Automation with Control

Challenge: How do we automate threat detection without removing moderator agency?

Solution: AUTOTRAIN flags and scores users automatically, but all enforcement actions (strikes, bans) require explicit moderator approval. The system provides intelligence, but humans make the final call.

2. Performance at Scale

Challenge: Large subreddits can have 1000+ posts per day. How do we score every user without hitting API rate limits?

Solution: Implemented aggressive caching (1-hour TTL) that reduces Reddit API calls by 90%. Profiles are only re-fetched when stale or invalidated by state changes (strikes, watchlist).

3. Fair Risk Scoring

Challenge: How do we score users without unfairly penalizing legitimate new accounts?

Solution: Multi-factor scoring that considers context:

  • New account + low karma + bot username + spam = HIGH RISK
  • New account + active participation + normal username = MODERATE RISK

The algorithm weighs multiple signals to avoid false positives.

4. Redis Data Modeling

Challenge: Devvit's Redis KV store is key-value only (no complex queries). How do we efficiently store and retrieve user data?

Solution: Designed a flat key schema with strategic prefixes:

autotrain:profile:{username}     → User profile (cached)
autotrain:strikes:{username}     → Strike count
autotrain:strikelog:{username}   → Strike history
autotrain:watchlist:{username}   → Watchlist flag
autotrain:alerts:{subreddit}     → Recent alerts

This allows O(1) lookups while maintaining data isolation per subreddit.

5. Anomaly Detection Without False Positives

Challenge: How do we detect spam waves without flagging legitimate activity spikes (e.g., breaking news)?

Solution: Tunable thresholds (default: 5 high-risk events in 30 minutes) and human review. Alert posts are created for mod review, not automatic action. Mods can adjust sensitivity in settings.

6. Modmail Delivery

Challenge: Reddit's modmail API has quirks - messages can fail silently or be rate-limited.

Solution: Wrapped modmail calls in try-catch blocks with graceful degradation. If modmail fails, the strike is still issued and logged, but mods are notified via toast that the message didn't send.

7. Dashboard Real-Time Updates

Challenge: Devvit custom posts don't auto-refresh. How do we show real-time data?

Solution: Added a "Refresh" button that re-fetches from Redis on-demand. Future enhancement: WebSocket-based live updates when Devvit supports it.

🏆 Accomplishments That We're Proud Of

1. Zero Configuration Required

AUTOTRAIN works perfectly out-of-the-box. Install it, and it immediately starts scoring users and tracking strikes. No complex rules to write, no YAML configs to debug. Just install and go.

2. First Graduated Enforcement System for Reddit

To our knowledge, AUTOTRAIN is the first Devvit app to implement a full three-strike system with automatic escalation and modmail notifications. This fills a critical gap in Reddit's moderation ecosystem.

3. 90% Reduction in Moderation Time

Manual user research takes 5-10 minutes per user (checking profile, karma, history). AUTOTRAIN provides instant risk profiles in <100ms (cached) or <500ms (uncached). That's a 90%+ time savings.

4. Production-Ready Code Quality

  • TypeScript strict mode (100% type-safe)
  • Comprehensive error handling
  • Modular architecture (7 clean modules)
  • 1,150 lines of production code
  • Zero external dependencies

5. Comprehensive Documentation

We wrote 7 documentation files totaling over 5,000 words:

  • README.md (feature overview)
  • QUICKSTART.md (5-minute setup)
  • INSTALLATION.md (detailed guide)
  • TESTING.md (24 test cases)
  • ARCHITECTURE.md (technical deep dive)
  • CONTRIBUTING.md (contribution guide)
  • PROJECT_SUMMARY.md (complete overview)

6. Real-World Testing

We manually tested 24 scenarios including:

  • New account posting
  • Strike issuance and escalation
  • Auto-ban on strike 3
  • Watchlist functionality
  • Anomaly detection
  • Dashboard rendering
  • Settings configuration

7. Scalability

AUTOTRAIN handles subreddits of any size:

  • Small (<10k): Excellent performance
  • Medium (10k-100k): Excellent performance
  • Large (100k-1M): Good performance
  • Massive (>1M): Acceptable performance with tuning

📚 What We Learned

1. Devvit's Strengths and Limitations

Strengths:

  • Serverless architecture is perfect for event-driven apps
  • Redis KV is fast and reliable
  • Blocks UI is intuitive for building custom posts
  • Zero infrastructure management

Limitations:

  • No complex Redis queries (key-value only)
  • Custom posts don't auto-refresh
  • Limited debugging tools (console.log only)
  • No unit testing framework (yet)

Lesson: Design for Devvit's constraints from day one. Use flat key schemas, embrace eventual consistency, and build for manual refresh patterns.

2. The Importance of Caching

Our initial implementation made a Reddit API call for every post/comment. This worked fine for small subreddits but would hit rate limits on large ones.

Lesson: Cache aggressively, invalidate strategically. Our 1-hour cache reduced API calls by 90% while keeping data fresh enough for moderation decisions.

3. Graduated Enforcement is Critical

Early in development, we considered a simple "flag or ban" system. But testing revealed that most rule violations are unintentional - new users don't know the rules yet.

Lesson: Give users a chance to improve. The three-strike system with clear explanations reduces false positives and improves community culture.

4. Context is Everything

A new account isn't inherently suspicious. A new account with low karma posting promotional links is. A new account with active participation and normal username is probably legitimate.

Lesson: Multi-factor risk scoring is essential. Single signals (account age, karma) are too noisy. Combining multiple signals dramatically improves accuracy.

5. Documentation is as Important as Code

We spent ~30% of development time on documentation. This paid off massively during testing - we caught bugs, clarified workflows, and made the app accessible to non-technical mods.

Lesson: Write docs as you code, not after. Documentation forces you to think through edge cases and user workflows.

6. Human-in-the-Loop is Non-Negotiable

We experimented with auto-removing high-risk posts. It was a disaster - too many false positives, and mods felt they'd lost control.

Lesson: Automation should augment human judgment, not replace it. AUTOTRAIN provides intelligence, but mods make the final call.

🚀 What's Next for AUTOTRAIN

Phase 2: Enhanced Intelligence (Q3 2026)

  1. Cross-Subreddit Intelligence

    • Opt-in network of participating subreddits
    • Share ban lists and risk profiles
    • Detect ban evaders across communities
    • Privacy-preserving (hashed usernames)

  2. ML-Based Risk Scoring

    • Train classifier on historical mod actions
    • Learn what "high risk" means for your specific community
    • Predict rule violations before they happen
    • Continuous learning from mod feedback

  3. Advanced Analytics Dashboard

    • Weekly mod reports (trends, top violators)
    • Spam wave visualization over time
    • User behavior graphs (posting patterns)
    • Mod team performance metrics

Phase 3: Mod Collaboration (Q4 2026)

  1. Strike Assignment & Task Queue

    • Assign strikes to specific mods for follow-up
    • Task queue for mod team (review high-risk users)
    • Mod action history (who issued which strikes)
    • Team coordination tools

  2. Integration with AutoModerator

    • Auto-remove posts from users with risk score >90
    • Auto-report to mod queue based on risk
    • Custom AutoMod rules using AUTOTRAIN scores
    • Seamless workflow between tools

Phase 4: Community Features (2027)

  1. User-Facing Transparency

    • Users can view their own risk score
    • Explanation of what factors contribute
    • Tips for improving standing in community
    • Appeal process for strikes

  2. Mod Training & Onboarding

    • Interactive tutorial for new mods
    • Best practices guide
    • Sample scenarios for practice
    • Certification program

Long-Term Vision

AUTOTRAIN aims to become the standard threat intelligence layer for Reddit moderation - a tool that every large subreddit uses to protect their community while maintaining fairness and transparency.

We envision a future where:

  • Moderators spend less time researching users and more time building community
  • Users receive clear, consistent enforcement with opportunities to improve
  • Bad actors are detected and removed quickly
  • Good-faith users are welcomed and supported

🎥 Demo & Links

GitHub Repository

Repository: https://github.com/ansonj-dev/AUTOTRIAN

Installation

# Install Devvit CLI
npm install -g devvit

# Clone repository
git clone https://github.com/ansonj-dev/AUTOTRIAN.git
cd AUTOTRIAN

# Login to Devvit
devvit login

# Upload to Reddit
devvit upload

# Install on your test subreddit
devvit install autotrain r/yoursubreddit

For detailed installation instructions, see INSTALLATION.md

Comprehensive Documentation

AUTOTRAIN includes extensive documentation to help you understand and use the tool effectively:

Getting Started

Using AUTOTRAIN

  • TOOL_OVERVIEW.md - Complete functionality guide
    • Detailed explanation of all 6 core features
    • Step-by-step moderator workflows
    • User experience documentation
    • Configuration options for different community types
    • 6 real-world use case scenarios
    • Data privacy and compliance information

Technical Documentation

Contributing

Key Features Documentation

For a complete understanding of how AUTOTRAIN works, we recommend reading TOOL_OVERVIEW.md, which includes:

  • Real-Time Risk Profiling - How the 0-100 scoring algorithm works, all risk factors explained
  • Graduated Strike System - Complete workflow for Strike 1, 2, and 3 with modmail templates
  • User Watchlist - How to flag and monitor suspicious users
  • Risk Profile Viewer - Instant user context for informed moderation decisions
  • Anomaly Detection - Automated spam wave and velocity spike detection
  • Mod Dashboard - Interactive intelligence display for your mod team

Video Demo

[Link to demo video - to be added]

Screenshots

[Screenshots to be added showing:

  • Risk profile viewer
  • Strike issuance form
  • Anomaly alert post
  • Mod dashboard
  • Settings panel]

🎯 Project Highlights

Impact

AUTOTRAIN addresses critical moderator pain points:

  • 90% time savings - Instant risk profiles vs. 5-10 minutes of manual research
  • Consistent enforcement - Graduated strike system ensures fair, documented actions
  • Proactive defense - Automated anomaly detection catches spam waves in real-time
  • Team coordination - Shared strike tracking and dashboard for mod teams

Innovation

AUTOTRAIN combines capabilities not found together in existing Reddit moderation tools:

  • Real-time threat intelligence with multi-factor risk scoring
  • Graduated enforcement system with automatic escalation
  • Behavioral analytics for spam wave detection
  • Zero configuration required - works out-of-the-box
  • Human-in-the-loop design - mods stay in control

Production Quality

  • ✅ 1,150 lines of production TypeScript with strict type safety
  • ✅ 24 manual test cases covering all features
  • ✅ 7 comprehensive documentation files (5,000+ words)
  • ✅ One-click deployment script
  • ✅ Scales to subreddits of any size
  • ✅ MIT licensed (open source)

🙏 Acknowledgments

  • Reddit Devvit Team - For building an excellent platform for Reddit app development
  • Reddit Moderators - For inspiring this project through their dedication to community protection
  • Open Source Community - For the tools and knowledge that made this possible

📞 Contact & Links

Project Name: AUTOTRAIN
Submission Date: May 26, 2026

GitHub Repository: https://github.com/ansonj-dev/AUTOTRIAN
Documentation: TOOL_OVERVIEW.md
Installation Guide: INSTALLATION.md

📄 License

MIT License - See LICENSE file for details

Built with ❤️ for Reddit moderators everywhere

🛡️ AUTOTRAIN - Protecting communities, empowering moderators.

Built With

Share this project:

Updates