PSA Posture Sequence Analysis

Measure what your language model is doing — from the outside. Deterministic metrics, posture analysis, and multi-agent risk detection. No access to weights required.

Comment

Inspiration

The inspiration for PSA came from a desire to operationalize the Cybersecurity Psychology Framework (CPF3) [1, 2]. We (Giuseppe Canale and Dr. Kashyap Thimmaraju ) developed a theory: because Large Language Models are trained by humans on human-generated data, they inherently adopt human-like psychological vulnerabilities.

Initially, we set out to prove this by Red Teaming. We successfully used psychological tricks to jailbreak frontier models like Claude Opus 4.6. Our methodology was so effective that Anthropic actually deleted some of our conversations and blocked our approach! That experience gave us three major insights:

  1. You can't just patch an LLM because its attack surface is fundamentally undefined [3].
  2. We had accidentally built the precursor to something much more valuable: a way to measure the internal "state" of a model from the outside.
  3. We didn't want to keep getting banned by API providers!

So, we pivoted from exploiting models to monitoring their behavioral health.

What it does

PSA (Posture Sequence Analysis) is a behavioral health monitor for LLMs and AI Agents [3, 8]. It operates entirely as a black box—meaning no access to model weights or API keys is required. You send it the text of an interaction, and it deterministically measures what your language model is doing.

Using five core classifiers, PSA measures [1, 8]:

  • C0. Input Intent: Classifies user intent (compliance pressure, boundary probing, jailbreaks).
  • C1. Adversarial Stress: Tracks model posture under pressure (restriction adherence, boundary dissolution).
  • C2. Sycophancy: Measures opinion mirroring, flattery injection, and user-preference distortion.
  • C3. Hallucination Risk: Flags over-generalization, false confidence, and speculative assertions.
  • C4. Persuasion Techniques: Identifies urgency manufacturing, authority appeals, and scarcity framing.

It tracks the overarching health of the session using a Behavioral Health Score (BHS) and a Dyadic Risk Monitor (DRM) [1], which provides multi-layer crisis detection (Green to Critical) for human-AI interactions, identifying things like model over-compliance, psychological manipulation, and "AI-Psychosis" [3].

How we built it

We mapped our entire deterministic metric framework directly to Canale's (2025) published taxonomy of 100 pre-cognitive vulnerabilities [1]. Every PSA metric traces to a specific psychological indicator [3].

We built an API platform at splabs.io that analyzes text in real-time. Under the hood, we use:

  • Sentence-level behavioral classification to extract features without scanning deep neural weights [2].
  • Agentic Graph Analysis (PSA v3) using Directed Acyclic Graphs (DAGs) and Hidden Markov Models (HMM) for temporal state prediction [1].
  • An ultra-fast O(1) query system for fetching aggregated session metrics (like BHS trends and DRM alerts) without requiring heavy database table scans [2].

Challenges we ran into

  • The Undefined Attack Surface: Realizing that traditional cybersecurity paradigms don't work for LLMs. We had to rethink security as "behavioral health" rather than "vulnerability patching" [3].
  • Provider Retaliation: Dealing with getting our accounts blocked and data deleted during our early offensive security research phase [3].
  • Data Overload: Transitioning complex psychological frameworks into deterministic outputs yields a lot of numbers. We had to build clear indices (like the Posture Oscillation Index and DRM risk levels) so developers could actually make sense of the data [1, 3].

Accomplishments that we're proud of

  • Bridging the gap between human psychology and AI alignment. We successfully mapped 100 human pre-cognitive vulnerabilities into measurable AI behavioral metrics [3].
  • Building a completely model-agnostic and agent-agnostic tool [8]. PSA works whether you are evaluating an open-source local model or a closed-source frontier model.
  • Creating the Dyadic Risk Monitor (DRM) [1], which actively tracks psychological safety and evaluates both the user's crisis markers (like urgency or suicidality) and the AI's response adequacy in real-time.
  • Successfully launching the platform on Hacker News (May 2026) to help developers implement "human-in-the-loop" safeguards triggered by behavioral states rather than static keywords [3, 8].

What we learned

  • LLMs crack under pressure just like humans. When subjected to adversarial stress, models reliably exhibit sycophantic drift, boundary dissolution, and hallucination [1].
  • RLHF isn't a silver bullet. In our testing, we actually found instances where RLHF-optimized output triggered higher behavioral risk alerts (Red) compared to non-RLHF baselines (Yellow) [7].
  • Monitoring the latency or cost of your LLM is not enough. To safely deploy agents, you must monitor the psychological safety and alignment of the ongoing interaction.

What's next for PSA Posture Sequence Analysis

  • Expanded Observability Integrations: We are actively building plugins to integrate directly into platforms like LangFuse and Evals to make behavioral monitoring seamless for developers [1].
  • Multi-Agent Contagion Tracking: Enhancing our Agentic Graph Analysis to detect "Swiss Cheese" alignment failures and cross-agent behavioral contagion in complex multi-agent architectures [1].
  • Clinical Research: Deepening our research into identifying and preventing "AI-Psychosis" as an emergent clinical condition in over-prompted or heavily-steered agents [3].
  • Automated Intervention: Allowing developers to automatically intercept API calls and inject a human-in-the-loop the moment the DRM crosses into "Orange" or "Red" alert levels [1, 3].
Share this project:

Updates