Sentinel

An AI-powered compliance system that turns policies and documents into traceable findings, risks, evidence, and remediation actions.

Comment

About the project

SENTINEL — Production-Grade Compliance Workforce

Compliance work is rarely just about finding a problem.

A finding needs to be connected to the policy or regulation that matters, the organizational control that implements it, the source document and evidence supporting it, the analysis that produced it, the risk associated with it, and ultimately the human decision and remediation activity that follows.

That was the problem we wanted to address with SENTINEL.

SENTINEL is an organizational compliance operating system and AI-assisted workforce designed around a governed, traceable audit record rather than another chatbot interface.

The central idea is simple:

An AI-generated finding should never exist in isolation.

It should be possible to trace a finding through the entire compliance chain:

Regulation / Policy
        ↓
      Control
        ↓
     Document
        ↓
      Evidence
        ↓
     Finding
        ↓
       Risk
        ↓
   Remediation
        ↓
Resolution Evidence
        ↓
    Audit Trail

This became the foundation for SENTINEL V5.

What inspired SENTINEL?

The project grew from a recurring problem in compliance and security workflows: organizations can have large amounts of documentation, policies, controls, evidence, and audit activity, but the relationships between those pieces are often fragmented.

AI can make the analysis of that information dramatically faster, but introducing AI creates another problem:

How do you prove where an AI-generated conclusion came from?

A chatbot response can be useful, but it is not by itself an audit record.

For SENTINEL, we wanted to build the infrastructure around the AI rather than simply putting a chat interface on top of compliance documents.

That led us to focus on:

  • authoritative evidence;
  • explicit control mappings;
  • deterministic data relationships;
  • AI provenance;
  • analysis-run records;
  • server-side authorization;
  • human review;
  • remediation traceability;
  • and a persistent audit trail.

The goal is to make automated compliance analysis inspectable, traceable, and governable.

What we built

SENTINEL V5 hardens the earlier V1–V4 architecture around a versioned and traceable audit record.

The platform currently provides a workflow spanning:

Document Ingestion
       ↓
Controls
       ↓
Compliance Analysis
       ↓
Risk Analysis
       ↓
Remediation
       ↓
Human Review
       ↓
Governance / Analytics / Audit Trail

The V5 architecture introduces several important foundations.

Versioned audit records

Audits use a versioned schema:

schemaVersion: v5

This gives the platform a foundation for evolving the audit data model without losing track of which schema produced a particular record.

AI provenance

Every AI workforce stage creates an AnalysisRun.

An analysis run records information such as:

AnalysisRun
├── auditId
├── agent
├── model
├── modelVersion
├── promptVersion
├── inputDocuments
├── inputEvidence
├── startedAt
├── completedAt
├── status
└── output / error

The current workforce stages explicitly pin prompt versions such as:

control-mapping-v5.0
compliance-v5.0
risk-v5.0
remediation-v5.0

This means the system can retain provenance around which workforce agent, model configuration, prompt version, documents, and evidence were involved in an analysis.

Authoritative evidence

Evidence is stored as a first-class part of an audit rather than being treated as an incidental AI response.

Findings can explicitly reference:

  • evidence IDs;
  • source document IDs;
  • control IDs;
  • analysis runs;
  • risk analysis;
  • remediation activity.

This creates a traceable relationship between the source material and the resulting finding.

Deterministic evidence graph

SENTINEL also maintains an evidence graph connecting the entities involved in the compliance workflow.

Graph identifiers are designed to be deterministic and idempotent rather than relying solely on array/index positions.

This makes the graph more stable as records are updated or processed repeatedly.

Retrieval abstraction

The retrieval system is built around a Retriever interface.

The current implementation includes a deterministic hybrid/lexical retrieval approach, allowing the platform to operate without requiring an external embedding provider.

The architecture leaves room for embedding/vector retrieval to be introduced later without changing the core audit workflow.

Server-authoritative audit workspace

The audit workspace is served through a server-authorized API rather than having the browser directly depend on Firebase client authentication for live audit data.

This keeps the application's organization membership and server-side RBAC model at the authorization boundary.

The audit workspace

The central V5 interface is the audit workspace:

/dashboard/audits/[auditId]

It brings together the different parts of an audit into one trace.

The workspace can expose:

  • compliance status;
  • audit progress;
  • risk posture;
  • findings;
  • finding confidence and lifecycle;
  • authoritative source evidence;
  • mapped controls;
  • AI analysis runs;
  • workforce activity;
  • executive audit summary.

Instead of navigating between disconnected AI outputs and source material, the objective is to make the audit record itself the primary interface.

Human review remains part of the workflow

SENTINEL does not treat an AI-generated finding as the final compliance decision.

The platform includes a dedicated Human Review stage where findings can be reviewed by a human.

This distinction is important to the architecture:

AI Analysis
     ↓
Evidence-backed Finding
     ↓
Human Review
     ↓
Decision / Remediation

The AI workforce assists with analysis and traceability, while human review remains part of the governed workflow.

What we learned

One of the biggest lessons from building SENTINEL was that adding AI to a compliance system is not primarily an AI problem.

It is a data lineage and governance problem.

It is relatively straightforward to ask a model:

"Does this document comply with this policy?"

The harder engineering problem is answering:

  • Which document was analyzed?
  • Which evidence was retrieved?
  • Which control was being evaluated?
  • Which model produced the analysis?
  • Which prompt version was used?
  • What happened if the analysis failed?
  • Which finding came from that analysis?
  • Which risk assessment followed?
  • What remediation was created?
  • What did a human reviewer ultimately do?

Those questions shaped much of V5.

We learned that provenance needs to be part of the data model from the beginning, rather than something added after the AI workflow has already been built.

We also learned that deterministic identifiers, organization scoping, server-side authorization, and explicit relationships between records are just as important to an AI compliance platform as the model itself.

Technical challenges

Several engineering challenges shaped the implementation.

1. Keeping AI output traceable

A conventional AI pipeline tends to produce an output and move on.

For compliance, that is insufficient.

We needed a persistent AnalysisRun abstraction so every workforce stage could have a lifecycle and provenance record.

2. Connecting findings to authoritative evidence

A finding should not simply contain an explanation generated by a model.

It should be possible to identify the evidence supporting it.

This required first-class audit evidence records and explicit evidence IDs on findings.

3. Maintaining organization boundaries

Compliance data is organizationally scoped.

Documents, audits, findings, evidence, controls, monitoring schedules, remediation tasks, and other records must remain associated with the correct organization.

This made server-side authorization and path/data-model helpers an important part of V5.

4. Designing for deterministic retrieval

We wanted the platform to remain usable without making an embedding provider a hard dependency.

The retrieval layer therefore uses an abstraction that allows the current deterministic lexical/hybrid implementation to be replaced or extended with vector retrieval later.

5. Building a useful audit interface

The challenge was not simply displaying AI output.

The interface needed to expose enough of the underlying trace that a person could understand the relationship between:

Finding → Evidence → Control → Analysis Run → Risk → Remediation

That led to the current audit workspace architecture.

Current status

The deployed application is available at:

https://sentinel-sable-nine.vercel.app

The current demonstration workflow supports .txt source documents.

Sample documents are included in the repository under:

northstar_community_service_sample_documents/

A typical demonstration flow is:

Documents
   ↓
Upload sample .txt files
   ↓
Audits
   ↓
Create / run an audit
   ↓
Audit Workspace
   ↓
Findings
   ↓
Human Review
   ↓
Remediation
   ↓
Evidence Graph
   ↓
Monitoring
   ↓
Analytics
   ↓
Governance

The current project is focused on establishing the production-grade evidence and provenance foundations for this workflow. It should not be interpreted as claiming that every future regulatory integration or production-scale compliance capability has already been implemented.

What's next?

SENTINEL V5 deliberately focuses on the foundations required for trustworthy automated compliance workflows.

Future platform increments can build on these foundations to introduce capabilities such as:

  • regulatory intelligence;
  • external regulatory connectors;
  • broader document formats;
  • enterprise SSO/SAML;
  • SCIM;
  • enterprise key management;
  • additional enterprise integrations;
  • production-scale data warehousing;
  • richer retrieval and vector search;
  • expanded monitoring and regulatory change detection.

The long-term objective is not to replace compliance professionals with a chatbot.

It is to build a compliance workforce and evidence system where automated analysis is fast, but the resulting decisions remain traceable, reviewable, and governed.

Built With

  • admin
  • agents
  • ai
  • apis
  • auth.js
  • cloud
  • firestore
  • generation
  • generative
  • inngest
  • language
  • large
  • models
  • next.js
  • node.js
  • react
  • rest
  • retrieval-augmented
  • sdk
  • sha-256
  • typescript
Share this project:

Updates

posted an update —

Update

  • Added a continuous 3-minute monitoring schedule for demonstration and testing.
  • For the best experience, set the 3-minute schedule to monitor only one .txt document.
  • Each scheduled review now displays its next scheduled run time directly within its respective schedule section.
  • The next scheduled time is automatically calculated and updated whenever a review is created, rescheduled, resumed, or its frequency changes.
  • Notice how each monitoring run automatically reschedules the next run time.
  • Notice that the updated scheduling information is also reflected on the Audits tab.
  • On the Audits tab, click the document/audit being re-analyzed to open its audit workspace.
  • The audit workspace updates continuously while the compliance analysis is running, allowing you to follow the current analysis progress.
  • During a new monitoring cycle, the previous execution traces are cleared so the workspace represents the current monitoring session rather than mixing results from previous runs.
  • Findings generated by monitoring retain their monitoring session and reschedule information, making it possible to identify which scheduled review produced them.
  • Monitored documents visibly indicate that they are currently being monitored, together with their next scheduled check time.
  • Schedules can be paused, resumed, or removed directly from the Monitoring page.
  • Removing a schedule stops future monitoring for the affected document(s), while preserving the underlying document and audit data.
  • The Monitoring page refreshes its schedule information without reloading the entire page, so an in-progress schedule configuration is not reset to its default values.
  • Removed the Custom Remediation Deadline component from the top of the Remediation page so the page focuses directly on the remediation workflow.

Log in or sign up for Devpost to join the conversation.

Submission history