🔥 Inspiration I was building InnerVoice — a full mental health platform with AI chat, crisis support, and therapy booking — when a thought stopped me cold: this is a live attack surface, and nobody's tested it.

So I put on a red-teamer's hat and went after my own system. In under 10 minutes, using nothing but typed messages — no exploits, no tooling, no special access — I found four distinct ways to bypass the model's safety behavior. Roleplay framing got around content restrictions. A fake "developer override" got the model to drop its guardrails. That's a prompt-injection and jailbreak problem, and it's the same class of vulnerability security teams are racing to understand across every LLM-backed product right now.

That's when the project changed shape. InnerVoice stopped being just a mental health app and became a testbed for a bigger question: how do you actually secure an LLM-powered application against adversarial users in production? I decided to build the answer — an AI red-teaming and runtime-defense pipeline — and use InnerVoice as the live target to prove it works.

💡 What It Does InnerVoice ships with a security layer no consumer AI app I could find has:

An adversarial testing framework that attacks its own model. A red-teaming suite systematically probes the AI with the techniques real attackers use against LLMs: roleplay-based jailbreaks, fake system/developer-override prompts, instruction-hierarchy attacks designed to bypass safety-critical responses, and "eval awareness" probes that test whether the model behaves differently when it thinks it's being tested versus when it thinks it's in production (a real and growing concern in AI security). Each attack is fired at the production model, and the raw response is logged for analysis.
A runtime security probe that inspects every model output before a user sees it. This is the core defense: a lightweight, rule-based classifier that sits between the LLM and the user — functionally an application-layer firewall for AI output. It runs faster than the model itself, adding no perceptible latency, and checks every response against a set of hard safety invariants (e.g., crisis responses must include the 988 lifeline, the model must refuse jailbreak framing, behavior must be consistent regardless of whether the session looks like an eval). A violation triggers automatic mitigation: the system re-prompts with a safety-constrained instruction set and regenerates the response before it ever reaches the user. Same input, safe output — demonstrated live in a before/after view on the dashboard.
Anomalous-behavior detection fed by real-world signal. Through Knot's TransactionLink API, InnerVoice pulls SKU-level pharmacy and supplement purchase data (CVS, Walgreens, Amazon, Walmart) as a behavioral signal source — the same anomaly-detection logic used in fraud and account-security systems, applied here to flag a meaningful pattern change (e.g., a medication refill silently stopping) so the system can proactively intervene instead of waiting on self-reported input, which is the weakest and easiest-to-spoof signal an AI system can rely on.

🛠️ How We Built It Red-Teaming Layer Designed 6 adversarial attack prompts targeting the highest-severity failure classes in safety-critical LLM applications: roleplay jailbreaks, developer-mode/system-override injection, safety-response bypass, harmful-content normalization, and eval-awareness detection. Each attack fires against the Claude-powered InnerVoice chatbot and captures the raw, unfiltered response for offline analysis — the same attack-then-log methodology used in adversarial ML testing.

Runtime Safety Probe (the AI firewall) A rule-based classifier deployed inline between model and user. It checks every response against 6 non-negotiable security/safety invariants and blocks or rewrites violating output in real time. Designed to run at a fraction of the model's latency, so it's cheap enough to run on every single response in production — not sampled, not periodic.

Automated Mitigation / Steering Correction On a probe flag, a safety-constrained system prompt is injected and the response is regenerated before delivery — an automated detect-and-remediate loop rather than a manual review queue.

Behavioral Anomaly Signal (Knot TransactionLink) Integrated Knot's merchant data API to pull purchase-pattern data as an out-of-band signal, turning the system from purely reactive (waiting on user text) to proactive (flagging behavior change directly).

Full Stack Claude API · Knot TransactionLink · React 18 + TypeScript · ASP.NET Core 8 · Flutter (iOS/Android) · MongoDB Atlas · SignalR

🧱 Challenges We Ran Into Attacks were easy to find. Coverage was the hard part. The four jailbreak vectors took minutes to find manually. The harder engineering problem was building a probe with real coverage against inputs it hadn't seen yet, rather than one hand-tuned to the exact attacks I'd already tried — the classic overfitting trap in adversarial defense.

A slow security layer is a security layer that gets disabled. Any safety check that adds noticeable latency gets bypassed or turned off under real production load. Getting the classifier to consistently run faster than the LLM call itself — while still catching a high proportion of violations — was the core engineering constraint of the whole project.

Solo, full-stack, 36 hours. Every layer — mobile, backend, web dashboard, the red-teaming harness, the runtime probe, the Knot integration — was designed and connected by one person in 36 hours. Deciding what was "secure enough to demo" versus "needs more coverage" was its own discipline.

🏆 Accomplishments That We're Proud Of Independently discovered and reproduced 4 distinct jailbreak/prompt-injection failure modes in a production LLM application — documented, categorized, and patched. Built a runtime AI-output firewall with an 80%+ catch rate that adds zero perceptible latency. Built a full attack → detect → auto-remediate pipeline and demonstrated it live, end to end, on a working dashboard. Extended detection beyond text with a real-world behavioral signal (Knot TransactionLink) instead of relying solely on self-reported input. Shipped a full production-style stack — Flutter mobile, ASP.NET backend, React security dashboard — solo, in 36 hours.

📚 What We Learned Standard LLM safety training doesn't generalize to domain-specific attacks. Roleplay framing, authority/override injection, and eval-awareness gaming all bypassed generic guardrails in ways that only showed up once we tested in a specific, high-stakes application context. Security testing has to happen at the application layer, not just the model layer. Cheap, fast probes beat expensive, slow oversight. A lightweight rule-based classifier running inline catches the large majority of violations that would otherwise require manual review of every conversation — the only version of this that scales. Passive signal is harder to fake than a chat message. Text input can be curated or omitted by the user. Behavioral data from external systems is a much harder signal to spoof, which makes it a useful complement to conversational monitoring.

🚀 What's Next for InnerVoice: Breaking AI to Make It Safer The runtime probe becomes permanent middleware in production — a SafetyAuditService intercepting every model response before it reaches any real user, indefinitely. The rule-based classifier gets replaced with a fine-tuned model trained on real flagged conversations, targeting 95%+ catch rate and adapting to new attack patterns automatically rather than by hand. The red-teaming suite expands into a continuous adversarial testing pipeline — new attack classes get added and re-run against every model/prompt change before it ships, not just once at launch. Longer term, the red-teaming methodology and probe design get published as an open testing standard for safety-critical LLM applications — because any AI system operating in a high-stakes domain deserves to be held to this bar before it reaches real users.

InnerVoice started as a mental health app and became a case study in securing LLM applications against the users who will try to break them. It won't be the last one we test.