RedressCI

RedressCI is the remediation platform that turns reported AI failures into privacy-safe, evidence-backed regression tests, verifies fixes, and prevents them from returning in production.

Comment

Inspiration

When an AI system gives someone an unsafe, inaccessible, or misleading answer, the experience usually becomes a support ticket, screenshot, or incident report. The affected person rarely learns whether the problem was reproduced, whether it was fixed, or whether the same failure can happen again.

Traditional incident databases preserve what happened. Evaluation platforms help engineering teams test datasets they already own. RedressCI connects the missing middle: it turns an affected person’s experience into a privacy-safe, evidence-backed regression test, proves that the test catches the failure, and carries that protection into CI.

Our goal is simple: one person’s experience should protect the next person.

What it does

RedressCI is an AI remediation and continuous-assurance platform.

It guides a reported failure through four stages:

  1. Report the failure
    A reporter submits an interaction using text, private evidence, or browser voice dictation. The original evidence starts private and remains separate from anything developers can access.

  2. Review the evidence
    RedressCI proposes a redacted case, structures the incident, identifies unanswered questions, and helps discover relevant evidence. Human reviewers approve privacy, evidence, expected behavior, and consent.

  3. Prove the fix
    The approved case compiles into a portable regression test containing deterministic checks and GPT-5.6 semantic grading. RedressCI requires comparative proof: the known-broken response must fail and the corrected response must pass.

  4. Prevent regression
    Developers can export the privacy-safe evaluation into GitHub Actions or another CI system. If the harmful behavior returns after a model, prompt, retrieval, or application change, the runner exits non-zero and can block the release.

RedressCI carefully distinguishes two levels of proof:

  • Evaluation verified means the test successfully distinguishes reviewer-approved recorded broken and corrected responses.
  • Verified fixed requires running that evaluation against a configured live deployment and producing a separate signed deployment proof.

Reporters receive a private status link, a remediation timeline, consent controls, and a tamper-evident Redress Receipt describing exactly what was proven.

How we built it

RedressCI is built as a full-stack TypeScript application with a React and Vite interface, an Express API, portable evaluation files, and a standalone CI runner.

GPT-5.6 is used for tasks where semantic understanding adds real value:

  • extracting interactions from text and screenshots using strict structured outputs;
  • transforming an unstructured experience into a reviewable incident;
  • surfacing uncertainty and unanswered questions;
  • discovering evidence candidates from privacy-approved content; and
  • grading semantic requirements that cannot be evaluated reliably with keyword matching alone.

Deterministic assertions remain local and deterministic. GPT-5.6 can propose and grade, but it cannot approve privacy, consent, evidence, expected behavior, or verified status.

We used Codex throughout development to inspect and evolve the architecture, implement the reporter-to-CI workflow, build the privacy and evaluation gates, create regression tests, diagnose product issues, and prepare the application for deployment.

The platform also includes:

  • role-enforced privacy boundaries;
  • encrypted private artifact storage;
  • evidence-to-assertion provenance;
  • immutable grader-policy hashes;
  • signed receipts and proof bundles;
  • allowlisted deployed-system adapters;
  • privacy-safe aggregate failure detection;
  • reporter consent withdrawal;
  • GitHub Actions workflow generation; and
  • a resettable synthetic demonstration workspace.

The application is containerized with Docker, deployed on Render, and continuously tested through GitHub Actions.

Challenges we ran into

The hardest challenge was making every product claim truthful.

A recorded corrected response is not proof that a deployed system is fixed. We created separate states and proof artifacts for evaluation verification and live deployment verification so the interface never overstates what was tested.

Privacy was another architectural challenge. A developer needs enough information to reproduce a failure without automatically gaining access to the reporter’s identity, original transcript, or private artifacts. We enforce this separation on the server rather than treating redaction as a visual-only feature.

We also needed to prevent generated evaluations from encoding unsupported expectations. Every assertion must cite reviewer-approved evidence, and changing that evidence invalidates dependent evaluations and returns them to review.

Finally, model-based grading introduces uncertainty. We combine deterministic assertions with GPT-5.6 semantic grading, preserve a shared immutable grader-policy identifier, support inconclusive outcomes, and require the same evaluation to run against both broken and corrected responses.

Accomplishments that we're proud of

We are proud that RedressCI is a working remediation loop rather than a collection of disconnected AI features.

The hosted product demonstrates:

  • private failure reporting;
  • human-approved redaction and evidence;
  • portable evaluation compilation;
  • broken-versus-corrected comparative proof;
  • live deployed-target verification;
  • signed receipts and proof bundles;
  • private reporter follow-up;
  • GitHub CI protection; and
  • privacy-safe trend analysis.

The application currently passes 37 automated tests covering privacy gates, role boundaries, evidence provenance, compiler behavior, comparative validation, proof integrity, reporter links, deployed verification, routing, voice-input behavior, and CI exports.

Most importantly, the affected person remains connected to the engineering outcome instead of disappearing after submitting a report.

What we learned

We learned that the difficult part of responsible AI evaluation is not simply generating more test cases. It is establishing why an expected behavior is legitimate, protecting the person who reported the failure, proving that the evaluation detects the intended problem, and communicating closure without exaggeration.

We also learned that AI works best here as a bounded collaborator. GPT-5.6 accelerates extraction, structuring, discovery, and semantic evaluation, while explicit human approval protects the decisions that require authority and accountability.

Portable evaluations are also essential. RedressCI becomes more useful when it complements existing engineering systems instead of requiring organizations to replace their current observability, evaluation, or CI platforms.

What's next for RedressCI

The next milestone is a design-partner pilot with organizations operating public-interest or high-impact AI systems.

We plan to add:

  • managed organizational identity and reviewer assignment;
  • PostgreSQL-backed production persistence;
  • regional object storage and managed encryption;
  • expanded GitHub, GitLab, Jira, Linear, Slack, and Teams workflows;
  • multilingual reporting and accessibility validation;
  • notification delivery and remediation service-level objectives;
  • governed community evaluation packs; and
  • independent verification by domain and accessibility experts.

RedressCI’s long-term vision is a remediation network where reported AI failures become durable, evidence-backed protections that can travel across products, organizations, and releases.

Built With

Share this project:

Updates