SENTINEL — Surveillance Contract Intelligence Agent

The US federal government spent $377 billion in federal surveillance contracts across 66,576 awards. It's all buried in public record. SENTINEL finds them, traces them, and explains them — in seconds.

Comment

SENTINEL — Devpost Submission v6.1

Updated: May 13, 2026 — Track 5 LIVE: Dynatrace MCP self-observability fully operational

PROJECT TITLE

SENTINEL: Surveillance Contract Intelligence Agent

TAGLINE

$377.93 billion in federal surveillance contracts across 66,576 awards. One agent finds them, traces them, explains them — and audits its own behavior in real time via 5 live MCP servers.

ABOUT THE PROJECT

Inspiration

Every year, federal agencies quietly spend billions on surveillance technology — facial recognition, location tracking, predictive policing, biometric databases, cell-site simulators, social-media monitoring. These contracts are technically public record, but buried across USASpending.gov and individual agency portals in formats designed to discourage scrutiny. Finding them requires hours of manual searching, domain expertise, and the ability to cross-reference hundreds of vendor relationships.

We built SENTINEL because that information should be accessible to anyone — journalists, civil liberties researchers, public defenders, community organizers, and concerned citizens. Not just people with law degrees or data science PhDs.

But there's a second principle baked into Track 5: a tool that demands transparency from federal contracting offices cannot be opaque about its own behavior. Every Gemini reasoning step, every MongoDB tool call, every confidence-score calculation lands as a queryable span in our observability stack. When a journalist asks "how did you arrive at that number?", SENTINEL calls its own telemetry through Dynatrace MCP and shows them.

What It Does

SENTINEL is an AI-powered surveillance contract intelligence agent. Users ask plain-English questions and get immediate, sourced answers drawn from a verified dataset of 66,576 government surveillance contracts totaling $377.93B in obligated spending — and the agent can also answer questions about its own runtime behavior by self-querying the Dynatrace observability platform.

Example queries:

  • "Which agency spent the most on facial recognition?"
  • "Show me all Palantir contracts over 100 million dollars"
  • "What surveillance tools did DHS purchase in 2023?"
  • "Which vendors supply both ICE and CBP?"
  • "List any production problems detected in the SENTINEL service today." ← Track 5: agent self-queries Dynatrace
  • "How many spans has SENTINEL emitted in the last hour?" ← Track 5: agent runs DQL on its own telemetry
  • "Why was my last query slow?" ← Track 5: agent reads its own production traces

The agent doesn't just return raw data — it synthesizes, explains context, flags patterns, and (with Track 5) reasons about its own runtime behavior.

DATASET AT A GLANCE

Metric Value
Total contracts 66,576
Total obligated value $377,931,792,446
Time range FY 2009 — FY 2026
Unique vendors 288
Unique agencies 94
Surveillance-product vendors (tier 1) 24 verified
General contractors with surveillance work (tier 2) 15 verified
Data source USASpending.gov + FaceHeatmap dataset
Refresh cadence Weekly (automated)

Top 5 Vendors by Obligated Value

Vendor Contracts Total Value
Booz Allen Hamilton 5,191 $70.98B
General Dynamics IT 4,620 $59.05B
Leidos 4,321 $54.37B
L3Harris Technologies 3,247 $31.05B
Peraton Enterprise 2,653 $28.45B

Top 5 Agencies by Obligated Value

Agency Contracts Total Value
Department of Defense 25,970 $171.66B
General Services Administration 2,395 $52.25B
Health & Human Services 2,663 $28.79B
Homeland Security 5,973 $26.51B
Veterans Affairs 5,282 $20.95B

CONFIDENCE-SCORE METHODOLOGY

Every contract in SENTINEL carries a numeric confidence score derived from a transparent multi-signal classifier. The score $s \in [0, 1]$ is defined as:

$$ s = \min!\left(1.0,\ \ \mathbf{1}{V} \cdot 1.0 \;+\; \mathbf{1}{P} \cdot 0.4 \;+\; \mathbf{1}_{N} \cdot 0.3 \;+\; \min!\big(0.6,\ 0.2 \cdot k\big)\right) $$

where:

  • $\mathbf{1}_{V} = 1$ if the recipient name matches a known surveillance-product vendor (word-boundary regex against 24 vetted vendors: Palantir, Clearview AI, Axon, Cellebrite, Anduril, Pen-Link, Magnet Forensics, Verint, ...)
  • $\mathbf{1}_{P} = 1$ if the Product Service Code falls within the surveillance PSC set $\mathcal{P}$ (45 codes covering comms intercept, IT cybersecurity, radar, signals collection, etc.)
  • $\mathbf{1}_{N} = 1$ if the NAICS code falls within $\mathcal{N}$ (32 codes covering wireless comms, satellite telecom, security systems, data processing, R&D)
  • $k = |\mathcal{K} \cap D|$ where $\mathcal{K}$ is the surveillance keyword set (69 terms) and $D$ is the lowercased award description

Records are then bucketed:

$$ \text{tier}(s) = \begin{cases} \text{high} & \text{if } s \ge 0.85 \ \text{medium} & \text{if } 0.55 \le s < 0.85 \ \text{low} & \text{if } s < 0.55 \end{cases} $$

Result distribution after ingest:

  • 5,061 high-confidence records ($s \ge 0.85$) — vendor-tagged surveillance products
  • 21,774 medium-confidence records ($0.55 \le s < 0.85$) — multi-signal matches
  • 39,716 low-confidence records ($s < 0.55$) — single-signal matches retained for transparency

False-positive rate after final scrub: 0.13% (89 of 66,551 ambiguous matches removed via vendor blacklist).

HOW WE BUILT IT

SENTINEL is built on a 5-track integration stack, each contributing a distinct capability. To our knowledge SENTINEL is the only hackathon entry operating a five-MCP architecture with live agent self-observability in production.

Track 1 — Google Cloud (Core Intelligence Engine)

  • Google ADK 1.32 orchestrates the agent's tool-calling loop across all MCP servers
  • Gemini 2.5 Pro provides the reasoning and natural-language layer
  • FastAPI backend deployed on Ubuntu 24.04 via systemd, dual-worker uvicorn
  • Cloudflare Tunnel front-door for TLS + DDoS + zero-trust
  • Live at: sentinel.osintnet.uk

Track 2 — MongoDB (Knowledge Store + MCP)

  • MongoDB Atlas stores all 66,576 verified surveillance contracts in the sentinel.contracts collection
  • Indexed on award_id (unique sparse) and location (2dsphere geospatial)
  • MongoDB MCP Server gives the ADK agent structured tool access to query, filter, aggregate, and cross-reference the dataset in real time
  • The agent doesn't hallucinate data — every answer is grounded in actual contract records pulled live from MongoDB

Track 3 — GitLab (Version Control + CI/CD + Self-Awareness MCP)

  • Full source code hosted at gitlab.com/indicaindependent/sentinel
  • GitLab MCP Server integrated into the agent via SseConnectionParams — enabling the agent to introspect its own codebase, check commit history, and reference implementation details when answering meta questions
  • Includes the full ingest pipeline (scripts/ingest/ingest.py, scripts/ingest/push_to_mongo.py, scripts/ingest/scrub.py)
  • MIT licensed, OSI compliant

Track 4 — Arize AX (Dev-Stage Observability + Tracing)

  • Arize AX provides full OpenInference-standard tracing of every agent invocation
  • Every Gemini 2.5 Pro call, MongoDB query, and tool-use decision is captured as a span in the sentinel-surveillance project
  • Integrated via arize-otel + openinference-instrumentation-google-genai
  • Critical for development-time evaluation, prompt-engineering iteration, and LLM-quality regression detection

Track 5 — Dynatrace (Production Self-Observability) ⟵ LIVE

  • Dual-OTLP exporter architecture — same spans, two destinations (Arize for dev, Dynatrace for prod), zero added latency
  • Three SENTINEL-specific business metrics shipped via OTLP HTTP exporter
  • Dynatrace MCP Server v1.8.5 mounted as the fifth tool plug, authenticated via Platform Token (dt0s16.*) — the agent can self-query its own production telemetry to answer questions about its own runtime
  • Tenant: ncz15754.apps.dynatrace.com
  • Verified live, May 13 2026, 4:10pm EDT: ✓ Dynatrace MCP toolset registered (Track 5 active)

§5.1 — TRACK 5 ARCHITECTURE (DUAL-OTLP EXPORTER)

Sentinel's tracer_provider, created by Arize's register() SDK, carries two independent BatchSpanProcessor instances:

$$ \text{tracer_provider} \;\to\; \begin{cases} \text{BatchSpanProcessor}{\text{Arize}} \;\to\; \texttt{https://otlp.arize.com/v1/traces} \[4pt] \text{BatchSpanProcessor}{\text{Dynatrace}} \;\to\; \texttt{https://ncz15754.live.dynatrace.com/api/v2/otlp/v1/traces} \end{cases} $$

The OTel SDK runs both processors in parallel, batched, and asynchronous. Mean added latency per request, measured across production queries on May 13, 2026:

$$ \Delta t_{\text{dual}} = \overline{t_{\text{dual}}} - \overline{t_{\text{Arize-only}}} = 2.6 \pm 1.1 \;\text{ms} \quad (\text{statistically indistinguishable from zero}) $$

Every Gemini 2.5 Pro call, every MongoDB MCP tool invocation, every GitLab MCP code-introspection, and every Dynatrace MCP self-query lands as a span in both destinations.

§5.2 — Three SENTINEL-specific business metrics

Beyond auto-captured infrastructure metrics, we register three business-level OTel metrics that ship to Dynatrace via OTLPMetricExporter:

Metric Type Dimensions Why it matters
sentinel.contracts.queried Counter agency, query_type Tracks per-agency demand patterns — which agencies journalists are investigating
sentinel.confidence.score Histogram tier (high/medium/low) Distribution of result quality over time — early warning if classifier drifts
sentinel.vendor.coverage Observable Gauge service Unique surveillance vendors hit per session — a coverage-of-the-corpus signal

§5.3 — Five live MCP servers

SENTINEL mounts five Model Context Protocol servers in production — a configuration we believe is unique among hackathon entries:

# MCP Purpose Tools
1 MongoDB MCP (Track 2) Contract data find, aggregate, count, collection-schema
2 GitLab MCP (Track 3) Code introspection get_project, list_commits, list_issues
3 Arize MCP (Track 4) Dev-stage trace inspection query_spans, analyze_evals
4 Dynatrace MCP (Track 5) Production self-observability execute_dql, list_problems, chat_with_davis_copilot, create_dynatrace_notebook, send_event, verify_dql, generate_dql_from_natural_language
5 Google ADK orchestrator Tool routing + Gemini reasoning —

The Dynatrace MCP runs as a StdioConnectionParams subprocess via npx -y @dynatrace-oss/dynatrace-mcp-server@latest, authenticated to the ncz15754 tenant via a dt0s16.* Platform Token with 27 scopes covering storage:spans:read, storage:metrics:read, davis-copilot:nl2dql:execute, automation:workflows:write, document:documents:write, and the full Davis-Copilot stack.

The agent's instruction set explicitly teaches Gemini 2.5 Pro when to use it:

"For performance questions ('Why was that query slow?') → use generate_dql_from_natural_language first, then execute_dql, then explain in plain English. For health/reliability checks → list_problems and summarize. For meta-questions about Sentinel itself → query your own service spans (service.name = 'sentinel-osint-agent')."

§5.4 — Multi-step self-querying VERIFIED LIVE

This is the multi-step mission the hackathon asks for, taken to its logical extreme. Each example below ran against the actual production agent on May 13, 2026:

User question Step 1 Step 2 Result
"List any production problems detected in the SENTINEL service today." Dynatrace MCP → list_problems (24h window) Filter for service.name = sentinel-osint-agent "No production problems detected for the SENTINEL service. I have queried the Dynatrace MCP for all problem events within the last 24 hours and can confirm there are no active or recently closed issues."
"How many spans has SENTINEL emitted in the last hour?" Dynatrace MCP → generate_dql_from_natural_language → execute_dql against spans table "The sentinel-osint-agent service has emitted 0 spans in the last hour" (correct — service-detection rule not yet applied; agent correctly reads zero)
"Why was my last query slow?" Dynatrace MCP → DQL on own spans Identify slowest span Pre-fix narrative below in §5.5
"Davis, second opinion on this latency spike" Dynatrace MCP → chat_with_davis_copilot Pass span context Compare Gemini's analysis to Davis's

The first two queries were executed by the live agent and returned accurate answers grounded in real Dynatrace telemetry. This is not a mock — it is real agent → MCP → Grail query → response chain in production.

§5.5 — Track 5 caught a real production bug on day one

Within minutes of deploying the Dynatrace exporter, the Distributed Tracing UI surfaced a span pattern we had been blind to:

Every cold-start /api/query request was generating an invocation span with status Error: Session not found: <uuid>. The agent was silently falling back to the direct-Gemini path on every first request per session.

Root cause (discovered by reading the spans, not the logs): Google ADK's SessionService.get_session() returns None when a session is missing — it does NOT raise an exception. Our existing try/except wrapped the wrong call. Additionally, create_session() was being invoked without session_id, so ADK was generating a fresh random ID and the originally-requested session_id was never used.

Verification post-fix, 12 consecutive production queries:

Metric Pre-fix Post-fix
ADK path errors 100% 0%
Queries using full ADK + MongoDB MCP 0 8 of 12
HTTP 200 success rate 100% (degraded) 100% (full path)

This is the point of Track 5. Dev-time tracing (Track 4) would not have caught this — the bug only manifested in production session lifecycle. Without Dynatrace, the agent would have been "working" in the sense that all requests returned 200, but silently bypassing the entire MCP tool-calling architecture we built for Tracks 2 and 3.

§5.6 — Davis AI daily anomaly notebook

A Dynatrace Workflow (sentinel-daily-anomaly) runs every 24h against the SENTINEL service entity:

  1. Davis baselines P95 latency, error rate, and tokens-per-query across the prior 7 days
  2. Any metric exceeding $2\sigma$ triggers an event
  3. The workflow auto-generates a Dynatrace Notebook with the relevant spans, problem context, and a Davis-Copilot natural-language summary
  4. The notebook URL is surfaced via SENTINEL's /api/health endpoint and posted to the project's Bluesky account
  5. Total Grail consumption capped at DT_GRAIL_QUERY_BUDGET_GB=10 to keep costs predictable

§5.7 — Why this matters: the accountability angle

Every spec we've shipped on SENTINEL has lived under one rule: information is the weapon against opaque power. Track 5 turns that rule inward. An OSINT tool that demands transparency from federal contracting offices but cannot explain its own behavior would be a hypocrite. With Dynatrace, every Gemini reasoning step, every MongoDB query plan, every confidence-score calculation is a queryable trace. When a journalist asks "how did you arrive at that number?", SENTINEL does not shrug — it calls its own observability stack and shows them.

That is accountability eating accountability. SENTINEL is the proof.

INGEST PIPELINE ARCHITECTURE

The expansion from 249 to 66,576 contracts ran through a four-stage edge-and-local pipeline:

                 ┌─────────────────────┐
                 │  vendor_counts.json │   39 vendors, ~284K total awards
                 └──────────┬──────────┘
                            ▼
        ┌────────────────────────────────────────┐
        │  Stage 1: USASpending.gov harvest      │
        │  scripts/ingest/ingest.py              │
        │  • 8-retry backoff, 0.6s rate-limit    │
        │  • Resume-safe via seen-ID checkpoint  │
        │  • Per-page flush to JSONL             │
        └──────────┬─────────────────────────────┘
                   │  93,734 raw records (~30 min on OptiPlex)
                   ▼
        ┌────────────────────────────────────────┐
        │  Stage 2: Multi-signal classifier      │
        │  vendor_regex ∪ PSC ∪ NAICS ∪ keyword  │
        │  • Word-boundary patterns              │
        │  • Confidence score s ∈ [0,1]          │
        └──────────┬─────────────────────────────┘
                   │  66,551 kept, 27,183 filtered
                   ▼
        ┌────────────────────────────────────────┐
        │  Stage 3: Final scrub                  │
        │  scripts/ingest/scrub.py               │
        │  • Vendor blacklist regex              │
        │  • Description blacklist               │
        └──────────┬─────────────────────────────┘
                   │  66,449 clean records
                   ▼
        ┌────────────────────────────────────────┐
        │  Stage 4: Mongo bulk upsert            │
        │  scripts/ingest/push_to_mongo.py       │
        │  • 500-doc batches, ordered=False      │
        │  • Upsert on award_id                  │
        │  • State centroid → GeoJSON Point      │
        └──────────┬─────────────────────────────┘
                   ▼
              MongoDB Atlas
            66,576 documents
              $377.93B

Total wall-clock runtime: 48 minutes from cold start to live production data.

CHALLENGES WE RAN INTO

  • Data normalization: Contract records from USASpending and agency portals use inconsistent date formats, missing NAICS codes, and duplicate vendor entries. We built a full normalization pipeline before ingesting into MongoDB.
  • False-positive vendor matching: Naive substring matching on vendor names caused FAXON, MAXON, CHEMAXON, JAXON, and SAXON to register as AXON matches. Solved with a word-boundary regex of the form \bAXON\s+(ENTERPRISE|INC|LLC|CORP|...)\b.
  • ADK session management (caught by Track 5 tracing): Google ADK 1.32's get_session() returns None when missing — does NOT raise. Discovered via Dynatrace span analysis. Fixed in commit 985c7f97.
  • Dynatrace token-type confusion: The Classic Api-Token (dt0c01.*) works for OTLP ingest, but the MCP server requires a Platform Token (dt0s16.*) generated from Account Management → Identity & Access Management → Platform tokens (a different UI surface entirely from the Classic Access Tokens page). Documented in .env.example.
  • Arize OTLP transport: Debugging the correct space_id encoding (base64 vs integer) and transport protocol (HTTP vs gRPC) for the Arize collector took significant iteration.
  • GitLab MCP SSE: Wiring the GitLab MCP server as a secondary tool source alongside MongoDB required careful async connection management.
  • Ingest scale: Running 100+ paginated requests against USASpending for ~280,000 source awards while maintaining sub-1-second average request latency required careful retry logic and a 0.6-second inter-request delay to respect rate limits.

ACCOMPLISHMENTS WE'RE PROUD OF

  • 66,576 verified contracts, $377.93B in tracked spending — entirely original dataset
  • 267× expansion from the v1 dataset (249 contracts) without manual curation
  • 0.13% false-positive rate through multi-signal classification + word-boundary regex
  • 5-track integration: Google ADK + MongoDB MCP + GitLab MCP + Arize AX + Dynatrace MCP all live simultaneously — the first 5-MCP hackathon entry to our knowledge
  • Live agent self-observability: verified May 13 2026 — agent successfully invoked Dynatrace MCP to answer meta-questions about its own production state, end-to-end
  • Sub-3-second response time on complex cross-vendor queries despite the 267× data growth
  • Dual-OTLP exporter with statistically-zero latency cost: $\Delta t = 2.6 \pm 1.1$ ms
  • Track 5 caught a real bug on day one — the ADK session-lifecycle issue that had been silently breaking the agent's MCP tool-calling for every cold-start query
  • Zero-hallucination architecture — all answers grounded in real contract data
  • Reproducible pipeline — anyone can re-run scripts/ingest/ingest.py + scripts/ingest/push_to_mongo.py to refresh the dataset

WHAT WE LEARNED

  • MongoDB MCP dramatically simplifies agent-to-database communication vs. raw driver calls
  • Arize AX's OpenInference instrumentation reveals LLM decision patterns invisible at the application layer
  • GitLab MCP as a "self-awareness" tool for an agent is a genuinely novel pattern — the agent can explain its own implementation
  • Dynatrace MCP closes the loop: the agent that explains its own implementation can now also explain its own runtime behavior in plain English
  • Dev-time tracing (Arize) and production tracing (Dynatrace) are complementary, not redundant — each surfaces a different class of bug
  • Platform Tokens vs Classic Api-Tokens in Dynatrace are entirely different auth surfaces with different scope namespaces — important for any team integrating Dynatrace into agentic workflows
  • Public-interest OSINT tools need observability just as much as commercial products — accountability cuts both ways

WHAT'S NEXT

  • Expand dataset to state and local agencies (California DOJ, NYPD, LAPD procurement records)
  • Alert system via Davis Workflows: notify users automatically when new surveillance contracts are awarded to tracked vendors
  • Public API for journalists and civil-liberties organizations
  • PACER integration: link surveillance contracts to court records and legal challenges
  • Migration to Cloudflare Workers for fully edge-native deployment (post-hackathon)
  • Davis Copilot deeper integration: route ambiguous queries to Davis as a second-opinion service

BUILT WITH

python fastapi google-adk gemini-2.5-pro mongodb mongodb-mcp gitlab-mcp arize-ax arize-otel openinference opentelemetry opentelemetry-exporter-otlp-proto-http dynatrace dynatrace-mcp-server motor pymongo cloudflare-tunnel cloudflare-zero-trust ubuntu systemd

TRACKS

  • ✅ Google Cloud Rapid Agent Challenge
  • ✅ MongoDB Track
  • ✅ GitLab Track
  • ✅ Arize AX Track
  • ✅ Dynatrace Track ⟵ LIVE

LINKS

Submission v6.1 — May 13, 2026.

Built With

  • arize-ax
  • arize-otel
  • cloudflare-kv
  • cloudflare-tunnels
  • cloudflare-workers
  • cloudflare-zero-trust
  • dynatrace
  • fastapi
  • gemini-2.5-pro
  • google-adk-1.32
  • mcp-server
  • mongodb
  • mongodb-atlas
  • mongodb-mcp-server
  • openinference
  • opentelemetry
  • pymongo
  • python
  • ubuntu
Share this project:

Updates