JanusState

Trust outcomes, not claims. JanusState independently checks system evidence behind AI-agent work, catches false completion claims, and produces explainable verdicts with integrity-checked receipts.

Comment

Inspiration

While I was exploring AI agents and business automation, I kept noticing the same risky assumption. If an agent receives a successful response from a tool, it may treat that response as proof that the actual business outcome also succeeded.

That is not always true. A payment service might accept a payment request while the payment itself remains pending. An AI agent could see the accepted request, confidently report that the entire workflow is complete, and move on to the next task.

That gap between what an agent claims happened and what the connected systems show actually happened inspired me to build JanusState. The main idea behind it is simple: trust outcomes, not claims.

What it does

JanusState is an independent verification layer for AI agents. Instead of trusting an agent's completion message or confidence score, it checks evidence from the systems where the work should have occurred.

When an agent reports success, JanusState:

  1. Treats the agent's report and confidence as untrusted input.
  2. Collects read only evidence from the relevant business systems.
  3. Normalizes the evidence while preserving its freshness and provenance.
  4. Evaluates a verification contract containing deterministic assertions.
  5. Produces a VERIFIED, PARTIAL, FAILED, or UNCERTAIN verdict.
  6. Generates an integrity checked Janus Receipt containing the contract snapshot, evidence references, results, and timeline.

I built an Agent Reliability Lab to demonstrate this idea using two fictional fulfillment scenarios.

In the honest completion scenario, the CRM, payment, inventory, and messaging systems all support the agent's claim. JanusState checks the evidence and returns VERIFIED.

In the premature payment success scenario, the agent reports COMPLETED because the payment request was accepted. JanusState independently checks the payment state and discovers that it is still pending. Seven required checks pass, one fails, and the final verdict becomes PARTIAL.

This is the main moment I wanted the project to demonstrate. The agent said the work was complete, but JanusState checked what actually happened.

JanusState also includes an AI assisted contract builder. A user can describe an outcome in natural language and receive a structured verification proposal. The user can review the proposal, edit every assertion, reorder or remove checks, add new checks, and submit the candidate for strict validation.

The AI generated proposal cannot activate a contract, create evidence, or determine a verdict. Human review and deterministic validation remain separate from the AI generation step.

How I built it

I built JanusState with Next.js, React, TypeScript, Zod, a PostgreSQL architecture prepared through Prisma, Vitest, Playwright, and the Google GenAI SDK.

I divided the verification engine into clear trust boundaries:

  • Versioned runtime schemas validate contracts, claims, evidence, results, and receipts.
  • Read only connectors represent CRM, payment, inventory, and messaging systems.
  • An evidence processing layer validates identity, freshness, availability, and provenance.
  • A deterministic evaluator handles bounded operators without unsafe type conversion.
  • A verdict policy aggregates only required assertion results.
  • Idempotent ingestion prevents duplicate work within the current running process.
  • The receipt layer uses canonical JSON and SHA 256 integrity verification.
  • Gemini Flash Lite generates bounded contract drafts and fictional agent claims.
  • JanusState's deterministic logic independently evaluates the evidence and decides the final verdict.

Codex with GPT-5.6 was central to the entire development process. I used both GPT-5.6 Terra and GPT-5.6 Luna inside Codex, along with different reasoning levels depending on the task.

For complex areas such as security boundaries, evidence handling, concurrency, idempotency, verdict policy, receipt canonicalization, and architectural reviews, I used higher reasoning settings. For focused interface work, styling, documentation, and smaller test updates, I used lighter reasoning settings.

Before building the application, I created a PRD, architecture document, rules document, design specification, task tracker, and timestamped engineering memory. These documents helped Codex understand the project boundaries and prevented different development sessions from drifting away from the original idea.

I did not ask Codex to build the entire application in one broad prompt. I divided the work into small phases. Each prompt described the exact task, important invariants, prohibited behavior, acceptance criteria, required verification commands, and where Codex should stop.

Terra and Luna helped me continue this structured workflow across architecture, implementation, testing, debugging, security reviews, and interface development. Codex also helped generate focused test cases, identify missing edge cases, diagnose failures, and keep the documentation synchronized with the actual implementation.

I still reviewed the decisions, ran the commands, tested the user flows, corrected failed assumptions, and decided what should or should not become part of the final product.

Challenges I ran into

The hardest challenge was making sure that an agent's claim never became evidence for its own verification.

The agent's confidence, explanation, transcript, and expected demo result had to remain completely separate from the deterministic verdict. JanusState had to decide the result only from the verification contract and independently collected evidence.

I also had to distinguish a business failure from an execution failure. A pending payment should produce an honest business verdict. An unavailable connector or malformed response should produce uncertainty instead of an invented answer.

Other difficult areas included structured AI output, resistance to prompt injection, strict capability allowlists, evidence freshness, idempotent replay, concurrent execution, canonical JSON, accessible interface states, and safe error handling.

I also spent a lot of time resolving Windows specific Node.js, Git ownership, development server, and Playwright issues. Working through these problems while keeping the architecture consistent under a strict hackathon deadline was challenging.

For this hackathon version, I intentionally used fictional connectors and storage that exists only inside the current running process. This makes the demonstration predictable and reproducible, but it also means runs and receipts are not yet durable across restarts or multiple server instances.

Accomplishments that I am proud of

I am most proud that JanusState catches the exact failure it was designed around. An AI agent confidently reports that a workflow is complete, while independent evidence shows that the payment is still pending.

I am also proud of building:

  • A complete pipeline from agent claim to evidence, verdict, and receipt.
  • Four clear verdict states instead of a misleading success or failure result.
  • Strict separation between AI generated content and deterministic policy.
  • Human reviewable contracts with capability constrained editing.
  • Evidence freshness, provenance, and citation validation.
  • Idempotent scenario execution and safe replay behavior.
  • Downloadable Janus Receipts with integrity verification.
  • Unit, integration, server, and browser test coverage.
  • An accessible and responsive Agent Reliability Lab that clearly demonstrates the trust boundary.

What I learned

The biggest thing I learned is that reliable agentic systems need more than better prompts or higher confidence scores. They need independent observation, explicit contracts, deterministic policies, and the ability to honestly return uncertainty.

I also learned that a successful API response does not necessarily mean the business outcome succeeded. Verification needs to inspect the resulting state instead of only checking the action that attempted to create it.

Working with Codex also taught me that prompt structure matters. Small tasks, explicit boundaries, required tests, and documented stopping points gave me much safer and more maintainable results than broad implementation requests.

Using GPT-5.6 Terra and Luna at different reasoning levels also helped me balance speed and depth. I could use more reasoning for architectural and security decisions, then use lighter settings for focused implementation and presentation work.

What's next for JanusState

My next goal is to turn this reference implementation into a production ready developer platform.

The roadmap includes:

  • TypeScript and Python SDKs for agent integration.
  • Real CRM, payment, inventory, and messaging connectors with read only access.
  • Durable PostgreSQL persistence for contracts, runs, evidence, and receipts.
  • Contract versioning, activation, and lifecycle management.
  • Webhooks and background verification workers.
  • Authentication, workspace isolation, rate limiting, and encrypted credential management.
  • Cryptographically signed receipts.
  • Carefully bounded semantic verification while keeping deterministic evidence and policy authoritative.

My long term vision is for JanusState to become the independent outcome verification layer that businesses can place between autonomous agents and the systems those agents operate.

Built With

Share this project:

Updates