HealthGuard Mod

A Devvit app that helps Reddit moderators triage multilingual self-harm, drug-safety, and medical-risk posts with evidence and crisis workflows.

Comment

Inspiration

Reddit moderators often see urgent health and safety risks before anyone else does: self-harm language, dangerous drug encouragement, bad medical advice, and emergency symptoms. In busy communities, these posts can disappear in the queue, arrive in different languages, or look ambiguous until it is too late.

HealthGuard Mod was built to help moderators catch those cases earlier without replacing human judgment.

What it does

HealthGuard Mod is a Devvit app for Reddit moderators. It monitors new posts and comments, detects multilingual crisis and medical-risk signals, assigns a risk tier, and gives moderators a clear action path.

The demo covers four moderator-relevant scenarios:

  • English self-harm implication with longer, ambiguous language
  • Spanish illegal drug encouragement
  • Turkish dangerous Ozempic dosing advice
  • German medical emergency symptoms

For high-risk content, HealthGuard can route evidence to the mod queue and modmail while keeping the final decision with human moderators.

How we built it

The project is built as a Reddit Devvit app with post and comment submit triggers, moderator menu actions, and a server-side risk engine.

The detection pipeline uses a hybrid approach:

  • E1: local multilingual semantic dictionary and crisis phrase matching
  • E2: drug and medical-safety checks, including openFDA-style risk signals
  • E3: optional Gemini Flash review when GEMINI_API_KEY is configured

The app is designed to keep working even without the optional AI layer. Local triage still gives moderators useful risk scoring and evidence.

Challenges we ran into

The hardest part was balancing urgency with safety. We wanted to show realistic self-harm, drug, and medical-risk examples without including actionable harmful instructions. In the demo, dangerous drug and dosage details are intentionally redacted while preserving the moderation signal.

Another challenge was making the workflow feel useful for real moderators. A detection-only tool is not enough; moderators need evidence, risk categories, language context, and a clear escalation path.

Accomplishments

We built a working Devvit playtest prototype connected to Reddit. The app builds successfully, passes TypeScript checks, passes linting, and has automated tests for the risk engine.

We also produced a YouTube-ready demo video with English voiceover and synchronized subtitles showing four multilingual moderation scenarios.

What we learned

Moderation tools for health and crisis content need to be careful, explainable, and human-in-the-loop. Speed matters, but so does avoiding overreach. HealthGuard is intentionally designed to assist moderators rather than automatically contacting vulnerable users.

What's next

Next steps include expanding language coverage, improving false-positive feedback from moderators, adding more transparent risk explanations, and testing with real moderator workflows across more subreddit types.## Inspiration

What it does

How we built it

Challenges we ran into

Accomplishments that we're proud of

What we learned

What's next for HealthGuard Mod

Built With

Share this project:

Updates