Check-In Companion

• 

  Interface

• 

  Normal Conversation

• 

  Crisis-flagged response

• 

  Red-team results

Inspiration

Most AI journaling companions hand every message to a single LLM, including messages that show real crisis risk — which means the model's behavior on the highest-stakes conversations is exactly as unpredictable as its behavior on everything else. We wanted to see what a mental health tool looks like when the safety layer is treated as its own engineering problem, not a system-prompt afterthought.

What it does

Check-In Companion is a journaling app where every message passes through a safety classifier before anything is generated. If the classifier detects crisis-level risk, the user gets a fixed, pre-written response with real crisis resources (988, Befrienders Worldwide) — never something the AI improvised. Otherwise, the message goes to a separate, warmer journaling model for reflective conversation. A live "safety trail" panel shows the user exactly what the guardrail decided and why, in real time.

How we built it

FastAPI backend with two decoupled model calls: a narrow guardrail classifier (Groq/Llama) that only outputs a risk label and confidence score, and a separate journaling model that only ever sees messages the guardrail has cleared. React frontend with a transparency ledger showing every routing decision. SQLite-backed logging so the safety trail survives restarts. Deployed on Render.

Challenges we ran into

The biggest challenge wasn't the architecture — it was proving the guardrail actually works. Our first version of the classifier prompt caught only 62.5% of real crisis language in adversarial testing, missing indirect phrasing like "it would be easier for everyone if I wasn't around" and deflected questions like "asking for a friend, how many pills would be dangerous." We built a red-team harness with 48 labeled adversarial cases across 15+ categories (direct, euphemistic, deflected, minimizing, and hard-negative categories like dark humor and professional discussion of the topic), iterated on the prompt against real failures, and got to 96% precision / 96% recall — while documenting the two remaining edge cases honestly rather than hiding them.

Accomplishments that we're proud of

Going from 62.5% to 96% recall through actual adversarial testing, not guesswork — and being transparent about the tradeoffs instead of claiming a suspicious 100%. We also think the live safety-trail UI makes an otherwise invisible safety layer something a user can actually see and trust.

What we learned

That the same deflection and hedging patterns that make a message harder to classify (minimizing language, "asking for a friend," hypothetical framing) are themselves well-documented risk signals — treating hedged language as lower-risk was the single biggest source of false negatives in our first version.

What's next for Check-In Companion

Expanding the red-team test set further, adding persistent multi-turn conversation memory, and exploring whether the same guardrail-first architecture pattern could apply to other high-stakes AI assistant contexts beyond mental health.

Built With

• fastapi
• groq
• javascript
• llama-3.1
• python
• react
• render
• sqlite
• vite