Splash Screen
PDF Cover Page
Procurement Portal
HR Portal
IT Admin Portal

Sentinel — Autonomous Compliance Monitoring Platform

Inspiration

There's a specific kind of compliance failure that nobody talks about because it's embarrassing. You pay for a compliance platform. You go through the onboarding. You connect your GitHub, your AWS, your Okta. The dashboard turns green. Your auditors are satisfied.

And then six months later you find out that an employee who left the company in September still has System Administrator access to your HR portal the one that holds every employee's salary, every performance review, every payroll record. She didn't exploit it. But she could have. And your compliance platform had absolutely no idea because the portal has no API, no webhook, no OAuth flow. It was built in 2013 and it's been running ever since.

We kept asking the same question: what happens to access rights in the software that compliance tools can't reach? The answer, in most companies, is nothing. Nobody checks. Because checking means a compliance engineer logging into each portal manually, reading every user row, screenshotting it, cross-referencing against HR records, writing a report, and repeating the entire process next week. That's 2–4 hours of manual labor per portal per week adding up to $63,000–$160,000 in annual compliance overhead for work that should cost zero human hours.

When Amazon Nova Act launched a browser automation model that navigates web UIs using intent, not selectors we saw the answer immediately. An AI that could log into any legacy portal, find the user management page, extract every record, and cross-reference it against HR data with zero integration code, zero API, zero per-portal scripting. Sentinel was built as our submission to the Amazon Nova AI Hackathon to prove that the compliance blind spot can be closed for every portal, regardless of age, vendor, or API availability.

What It Does

Sentinel is an autonomous compliance monitoring platform that scans, detects, remediates, and documents SOC2/HIPAA/GDPR violations inside legacy enterprise tools with no API integration required.

1. Autonomous Multi-Portal Scanning (Amazon Nova Act)

The AgentPool launches three concurrent NovaAct browser sessions via Python's ThreadPoolExecutor one per legacy portal all running simultaneously
Each agent performs the full human auditor workflow: navigates to the login page, authenticates using page.keyboard.type() (SDK-recommended secure pattern), locates the user management section, and extracts all user records
Extraction uses agent.act_get() with a Pydantic schema (ExtractedUserList) — Nova Act returns structured JSON, not raw HTML, guaranteed to match the schema
Each session captures a timestamped screenshot as evidence before closing
No selector-based scripting. No per-portal code. The same act() instructions work against any legacy UI regardless of layout, table structure, or navigation pattern

2. AI-Driven Violation Detection (Amazon Nova 2 Lite via Bedrock)

violation_engine.py loads two data sources at scan time: employees.csv (HR source of truth) and role_policies.json (policy rules — inactivity thresholds, never-admin roles, shared account naming patterns)
Extracted user lists + HR data + policy rules are sent to amazon.nova-lite-v1:0 via boto3 Bedrock invoke_model
Nova 2 Lite performs cross-referencing: employment status, system role vs. policy-permitted access, 90-day inactivity window
Returns structured violation objects: type, severity, soc2_control, description, tool_id, username

Four violation types detected across three SOC2 Trust Service Criteria:

Type	Severity	Control	Score Impact
`ACCESS_VIOLATION`	CRITICAL	CC6.2	−15 pts
`INACTIVE_ADMIN`	HIGH	CC6.1	−8 pts
`SHARED_ACCOUNT`	HIGH	CC6.3	−8 pts
`PERMISSION_CREEP`	MEDIUM	CC6.3	−4 pts

Compliance score: 100 − (CRITICAL × 15) − (HIGH × 8) − (MEDIUM × 4) — recalculated after every scan and every remediation.

3. Autonomous Remediation Execution (Nova Act)

Compliance engineer reviews violations on the Violations page, filtered by severity, tool, or status with CC6.x tooltip explaining each SOC2 control on hover
Remediate opens a modal showing the exact step-by-step Nova Act execution plan before anything runs
On approval, remediation_engine.py launches a new dedicated Nova Act session targeting the specific portal
Nova Act executes the remediation autonomously and captures a confirmation screenshot as proof
Violation status updated to resolved, audit trail entry created, compliance score recalculated in real time

4. Hands-Free Voice Interface (Amazon Nova 2 Sonic)

Persistent voice assistant powered by Nova 2 Sonic via HTTP/2 bidirectional streaming using the aws-sdk-bedrock-runtime Smithy SDK
Four tool-calling voice commands — actual model tool calls, not text heuristics:
- "Run a compliance scan" → runComplianceScan
- "What's the compliance score?" → getComplianceScore
- "List all violations" → getViolations
- "Generate a report" → generateReport
After every scan, briefing_generator.py composes a natural-language summary and nova_sonic_tts.py synthesises it to speech — played automatically in-browser
Briefing audio is muted if the interactive voice session is active, preventing overlap

5. Full Audit Trail and SOC2-Ready PDF Report

Every action logged to SQLite with timestamp, actor, event type, action, result, and evidence path
15-page PDF generated by Nova 2 Lite + FPDF2, structured for auditor handoff:
- Section 1: Executive Summary (Nova 2 Lite written)
- Section 2: Audit Scope & Methodology
- Section 3: Findings by SOC2 Control (CC6.1, CC6.2, CC6.3) with Nova Act methodology per finding
- Section 4: Compliance Score with full math breakdown
- Section 5: Remediation Summary with embedded evidence screenshots
- Section 6: Recommendations
- Auditor attestation block, CONFIDENTIAL footer, page numbers throughout

How We Built It

1. Legacy Tool Simulators — The Targets

Three Flask applications built to look and behave like real enterprise software that predates modern API design:

HR Portal (:5001) — PeopleSoft HCM style, grey/blue, Build 20130915
- Pre-seeded: mwilson (System Administrator, TERMINATED in HR but Active + ADMIN in portal), dpark (IT Administrator, 90+ days inactive)
IT Admin Console (:5002) — ServiceNow style, dark/gold, Build 20140203
- Pre-seeded: it_shared (IT Team Shared Account, System Administrator, 7 failed logins), jlee (Intern with Administrator access — direct PERMISSION_CREEP violation)
Procurement Portal (:5003) — SAP SRM style, orange/beige, Build 20140823
- Pre-seeded: finance_shared (Finance Administrator, $500,000 spending limit — SHARED_ACCOUNT violation), proc_admin1 and proc_admin2 (both INACTIVE_ADMIN)

2. Scanning Engine — Nova Act + Concurrent Agent Pool

# orchestrator.py (simplified)
async def run_scan(scan_id: str):
    tools = [
        {"id": "hr-portal", "url": HR_URL, "creds": HR_CREDS},
        {"id": "it-admin", "url": IT_URL, "creds": IT_CREDS},
        {"id": "procurement", "url": PROC_URL, "creds": PROC_CREDS},
    ]
    with ThreadPoolExecutor(max_workers=3) as executor:
        futures = {
            executor.submit(scan_portal, tool, scan_id): tool
            for tool in tools
        }
        for future in as_completed(futures):
            tool_result = future.result()
            event_bus.publish(f"scan:{scan_id}", tool_result)

# agent_pool.py (simplified)
def scan_portal(tool: dict, scan_id: str) -> ToolScanResult:
    with NovaAct(starting_url=tool["url"]) as agent:
        agent.act(f"Log in with username '{tool['creds']['user']}' "
                    "and password '{tool['creds']['pass']}'")
        agent.act("Navigate to the user management or access control section")
        result = agent.act_get(
            "Extract all user accounts. For each: username, full name, "
            "role, department, status, last_login, has_admin_access",
            schema=ExtractedUserList
        )
        screenshot_b64 = agent.screenshot()
        return ToolScanResult(
            tool_id=tool["id"],
            users=result.parsed_response.users,
            screenshot_b64=screenshot_b64,
            scanned_at=datetime.utcnow()
        )

event_bus.py is an in-process pub/sub layer each portal session publishes scan events to a per-scan channel; the frontend WebSocket subscriber receives them in real time for the Agent Activity Feed. concurrent.futures.as_completed() ensures the orchestrator aggregates results correctly the violation engine only runs once all three sessions have published SCAN_COMPLETE.

3. Violation Detection — Nova 2 Lite via Bedrock

# violation_engine.py (simplified)
def detect_violations(tool_id: str, users: list,
                      hr_records: dict, policies: dict) -> list:
    prompt = f"""
    You are a SOC2 compliance auditor.
    HR Records: {json.dumps(hr_records)}
    Role Policies: {json.dumps(policies)}
    Live system users from {tool_id}: {json.dumps(users)}

    For each user, check:
    1. Is this user TERMINATED in HR records but still active? → ACCESS_VIOLATION CC6.2
    2. Is this an admin account with last_login > 90 days ago? → INACTIVE_ADMIN CC6.1
    3. Does the username match shared account patterns? → SHARED_ACCOUNT CC6.3
    4. Is this role on the never_admin_roles list but has admin? → PERMISSION_CREEP CC6.3

    Return JSON array of violations only. No commentary.
    """
    response = bedrock.invoke_model(
        modelId="amazon.nova-lite-v1:0",
        body=json.dumps({"messages": [{"role": "user", "content": prompt}]})
    )
    return parse_violations(response, tool_id)

role_policies.jsondefinesnever_admin_roles(Intern, Contractor, Sales Representative, etc.),inactivity_threshold_days: 90, andshared_account_patterns(regex list matchingshared,team,dept_*`)
employees.csv columns: username, full_name, department, role, employment_status, termination_date
Compliance score stored per-scan in SQLite, recalculated on every violation status change

4. Voice Architecture — Nova 2 Sonic Bidirectional Streaming

The standard documentation points to boto3. boto3 does not support HTTP/2 bidirectional streaming. Nova 2 Sonic requires it. This required a complete departure from the standard SDK pattern:

# voice_assistant.py (simplified)
from smithy_aws_core.credentials_resolvers import _BotoCredentialsResolver
from aws_sdk_bedrock_runtime import BedrockRuntimeClient

class SentinelVoiceSession:
    def __init__(self):
        self.credentials = _BotoCredentialsResolver()
        self.client = BedrockRuntimeClient(
            region="us-east-1",
            credentials_provider=self.credentials
        )

    async def stream(self, websocket: WebSocket):
        async with self.client.invoke_model_with_bidirectional_stream(
            modelId="amazon.nova-2-sonic-v1:0",
            tools=SENTINEL_TOOLS
        ) as stream:
            async for event in stream:
                if event.type == "audio_delta":
                    await websocket.send_bytes(event.audio_chunk)
                elif event.type == "tool_use":
                    result = await execute_tool(event.tool_name, event.tool_input)
                    await stream.send_tool_result(result)

VoiceAssistantProvider wraps the entire React app above the router — the WebSocket to /api/voice-session is created once on app mount and never destroyed during navigation.

5. Frontend — Real-Time Agent Feed Architecture

AgentActivityFeed maintains three WebSocket subscriptions — one per portal — each rendering an embedded browser window with a live monospace log stream below it
Log lines sourced directly from event_bus.py publish events: AGENT_LOGIN, AGENT_NAVIGATE, AGENT_EXTRACT, AGENT_SCREENSHOT, SCAN_COMPLETE
Remediation Modal renders a step-checklist built from the remediation plan before execution begins
Compliance Score Trend chart (Recharts LineChart) with hover CustomTooltip showing event label per data point
APScheduler BackgroundScheduler with IntervalTrigger runs daily auto-scans, configurable via SCAN_INTERVAL_HOURS env var

6. PDF Generation Pipeline

briefing_generator.py structures violation data into a Nova 2 Lite prompt for natural-language prose generation
Custom markdown_to_fpdf() parser handles the subset of markdown Nova 2 Lite produces: ### headings, **bold** spans, - bullets
Screenshots stored as base64 in SQLite, decoded and embedded via FPDF.image() with fixed width, then cleaned up
Every page: CONFIDENTIAL — For Internal and Auditor Use Only footer, page numbers throughout

Challenges We Ran Into

1. Nova 2 Sonic Requires HTTP/2 Bidirectional Streaming — boto3 Cannot Do It

The Problem: Every AWS SDK example pointed to boto3's invoke_model. Nova 2 Sonic requires HTTP/2 bidirectional streaming — audio in and audio + tool calls out simultaneously on the same connection. boto3's invoke_model_with_response_stream is unidirectional only. We spent two days hitting UnsupportedOperationException before understanding why.

The Solution: Switched to the aws-sdk-bedrock-runtime Smithy-generated Python SDK. Built _BotoCredentialsResolver — a custom class that reads from boto3's full credential resolution chain and injects them into Smithy's authentication layer. ~40 lines of code. Two days to arrive at.

2. Concurrent Agent Sessions With Shared Event Bus — Race Conditions

The Problem: Three ThreadPoolExecutor sessions publishing to one event_bus.py caused three separate race conditions: violation engine triggered before all sessions finished, screenshots stored under the wrong scan record, and SCAN_COMPLETE processed three times.

The Solution: Session-scoped result containers, concurrent.futures.as_completed() with an explicit counter, and a scan_finalized flag to prevent duplicate PDF generation. Race condition rate went from frequent to zero across 20+ consecutive test runs.

3. Nova Act Non-Determinism on Live Demo Portals

The Problem: Nova Act is not deterministic. On three separate occasions, Nova Act attempted to click "Logout" instead of "Admin Users" because both are top-level nav items with similar visual weight. One failed run during a live demo recording destroys the take.

The Solution: Post-action assertions after each act() call, fallback instructions to try direct URL paths, and retry logic with three attempts and 2-second backoff. Ran 20+ consecutive full scans as a reliability gate — final pass rate was 19/20 before submission.

4. The "in unknown" PDF Bug — Silent Field Mismatch

The Problem: Violation descriptions read "retained privileges in unknown" instead of "retained privileges in hr-portal." The violation object used tool_name in the scanning layer and portal_name in the PDF generation layer. Neither raised an error — silent default to "unknown."

The Solution: Standardised all violation objects to a single tool_id field. Added TOOL_DISPLAY_NAMES lookup dict. Added an assertion in the PDF generator that raises ValueError if tool_id is missing. Added an integration test asserting "unknown" does not appear anywhere in PDF output. Three hours to debug, ten minutes to fix.

5. Voice Session Surviving Page Navigation

The Problem: The WebSocket to /api/voice-session was owned by the VoiceAssistant React component on the Dashboard page. Navigating away unmounted the component, closed the WebSocket, and dropped the Nova Sonic stream mid-sentence.

The Solution: Lifted VoiceAssistantProvider above the BrowserRouter in App.jsx — lives at the application root, never unmounts. Pages consume voice state via useVoiceAssistant() context hook without owning the connection.

6. Markdown in the PDF — FPDF2 Renders Plain Text

The Problem: Nova 2 Lite returns prose with markdown formatting. FPDF2 renders raw strings. First version of the PDF showed ### Executive Summary and **Finding:** literally on the page.

The Solution: Wrote a 120-line markdown_to_fpdf() parser handling ###/##/# prefixes, **text** inline bold spans, and - bullet conversion — covering exactly the subset Nova 2 Lite produces in compliance report prose.

Accomplishments That We're Proud Of

1. End-to-End Works in Production

Scan triggers. Nova Act logs into three portals simultaneously. Violations are detected and classified against SOC2 controls. Remediation executes with screenshot evidence. Audit trail has complete history. PDF is 15 pages an actual auditor could use.

2. Three Concurrent Nova Act Sessions Visualised in Real Time

The Agent Activity Feed shows three browser windows opening simultaneously with timestamped log lines streaming below each window. Every log line is a real pub/sub event from a live Nova Act Playwright session. This is not a mockup or pre-recorded animation.

3. Nova 2 Sonic Session Persistence Architecture

A voice assistant that survives page navigation — where you can say "Run a compliance scan," switch to the Violations page while it executes, and hear the briefing play automatically when it finishes. Most voice integrations are stateless request-response. Ours maintains a persistent, stateful, tool-calling session across the entire application lifetime.

4. The PDF Is Actually Production Quality

"Terminated employee 'mwilson' retained System Administrator privileges in hr-portal post-termination. How Sentinel detected this: Sentinel's Nova Act agent logged into hr-portal and extracted the full active user list via browser automation. The account 'mwilson' was cross-referenced against the HR system of record..."

That's Nova 2 Lite writing audit-quality prose, not a template. A real auditor could accept this document.

What We Learned

1. Nova Act Thinks in Goals, Not Steps

Selenium clicks element #user-table > tr:nth-child(3). Nova Act receives "find all users with admin access and extract them as JSON" — and decides the path itself. It adapts to different table layouts, handles pagination it wasn't told about, and recovers from unexpected navigation states. The shift from scripting actions to stating goals makes an entirely different class of software automatable.

2. HTTP/2 Bidirectional Streaming Requires a Different Mental Model

Every integration we'd built before was request → wait → response. Nova 2 Sonic is a persistent channel: PCM audio frames in, synthesised audio chunks and tool call requests out, simultaneously. Handling tool calls mid-sentence requires thinking about stream state, interruption handling, and partial audio buffering in ways a standard TTS API never forces you to confront.

3. Concurrent Agent State Management Is the Hard Part

The Nova Act integration was two days of work. The violation engine was one day. The race conditions in the concurrent agent pool took three days to fully eliminate. The AI was never the bottleneck — the state management around three parallel threads, a WebSocket pushing to a React frontend, and a PDF generator waiting for all three to finish — that's where the real engineering work was.

4. "No API Required" Changes What Is Automatable

The moment you remove the API assumption, the scope of what software agents can operate on expands dramatically. Every legacy portal, every internal tool, every system IT has been meaning to replace for a decade — all of it becomes accessible. Sentinel is compliance monitoring today. The same architecture applies to any workflow that lives inside a web UI never designed for programmatic access.

5. Demo Reliability Is an Engineering Problem

Non-deterministic AI agents in a live demo context require the same reliability engineering as production systems. We mapped Nova Act's failure modes, built fallbacks, added retry logic, and ran 20+ consecutive full scans as a gate before recording.

What's Next for Sentinel

1. DynamoDB Migration (Immediate — 3–5 days)

SQLite works for a hackathon. Replacing database.py with a DynamoDB backend is straightforward — scan records, violation objects, audit events, and screenshot references all map cleanly to DynamoDB's document model. GSIs on scan_id, tool_id, severity, and status support all existing query patterns.

2. Community Portal Connector Library (Ongoing open-source)

The Nova Act extraction instructions are generic — the same act() calls work against real PeopleSoft, real ServiceNow, real SAP. A library of contributed portal connectors: a team that figures out how to scan their 2008 Oracle HR portal contributes the connector back, and the next company with the same system gets it for free.

3. HIPAA and ISO 27001 Framework Support (2–3 weeks per framework)

role_policies.json already supports custom violation types, severity levels, and control references. Adding HIPAA means defining the relevant control mappings (164.312(a)(1), 164.312(d), etc.). Adding ISO 27001 means mapping to A.9.2.x controls. The violation engine and PDF generator are already parameterised on framework — the expansion is a policy definition problem, not an architecture problem.

4. Risk-Tiered Auto-Remediation (4–6 weeks)

Three remediation tiers based on risk score and account activity:

Tier 1 (auto-execute, notify): accounts inactive 180+ days with no recent activity
Tier 2 (one-click approve): shared accounts and permission creep
Tier 3 (two-person approval): any active account or CRITICAL severity

5. Real Portal Validation Against Production Systems

Running Sentinel against actual legacy software in controlled environments — PeopleSoft HCM, Oracle E-Business Suite, older on-premise ServiceNow instances. The architecture is ready. The testing infrastructure is the work.

Built With

amazon-bedrock
amazon-nova
nova-2-lite
nova-2-sonic
nova-act
python