Inspiration
Everyone at this hackathon is building agents that can reason over enterprise data. We asked a different question:
What happens when the agent is right, but the action it wants to take is still too dangerous to execute?
An AI agent can correctly identify a critical vulnerability and propose a remediation – but executing that remediation across 147 repositories without oversight could bring down production systems. The agent can propose. The agent cannot grant itself authority.
ARF Sentinel closes this gap by inserting an independent, Bayesian governance layer between the agent's proposal and execution.
What it does
ARF Sentinel is the execution control plane for enterprise AI agents. It combines three systems:
CRAFT – discovers enterprise dependency data (which repos depend on a vulnerable package) All SQL is generated through CRAFT's semantic layer; the agent never writes a query manually.
Nemotron (via Nebius) – reasons over the evidence and proposes a remediation action
ARF – independently evaluates risk, blast radius, evidence confidence, and policy constraints using Bayesian Expected Loss Minimisation, then decides: APPROVE, DENY, or ESCALATE
If the action is too risky, ARF doesn't just block it – it computes a counterfactual safe scope (e.g., “instead of 147 repos, upgrade only 32”) and guides the agent toward a bounded, lower‑loss action.
Demo walkthrough
The Streamlit app follows a cinematic, auto‑playing 8‑phase narrative:
- Mission Control – animated workflow diagram, 147 repos exposed, 63% blast radius
- CRAFT Investigation – live scanning progress bar
- Nemotron Proposal – agent proposes upgrading all 147 repos with 92% confidence
- ARF Interception – animated battle bar: confidence (purple) vs. ARF risk posterior (red)
- Bayesian Decision Engine – full Bayesian formula, expected loss chart, ARF ESCALATEs
- Execution Boundary – hard block: “BLOCKED BY ARF – Human approval required”
- Counterfactual Analysis – ARF finds a safe scope of 32 repos, risk drops from 82% → 37%, ARF APPROVEs
- Final Approval – download the full audit trail as a PDF
How we built it
Architecture
The agent pipeline is a LangGraph state machine with a mandatory ARF governance node – there is no edge from the agent’s proposal directly to execution.
Bayesian Governance
ARF uses Bayesian Expected Loss Minimisation, not deterministic thresholds.
Risk probability is computed from the blast‑radius score \(B \in [0,1]\):
$$R = 0.5 + 0.5 \cdot B$$
The posterior risk is then calculated via:
$$P(\text{Risk} \mid \text{Evidence}) = \frac{P(\text{Evidence} \mid \text{Risk}) \cdot P(\text{Risk})}{P(\text{Evidence})}$$
Expected loss for each action:
- APPROVE: \(R \cdot \text{cost}(B)\)
- DENY: \((1 - R) \cdot C \cdot \text{cost_deny}\)
- ESCALATE: policy‑weighted penalty
ARF selects the action with minimum expected loss, unless a policy rule forces DENY.
Counterfactual Engine
When ARF escalates, the counterfactual module reduces the scope until the blast‑radius score falls below 0.6, then re‑evaluates through ARF. This proves ARF is not a blocker – it's a safe‑action optimizer.
Tech stack
- Frontend: Streamlit (dark cyberpunk theme, responsive, auto‑play cinema mode)
- Orchestration: LangGraph (typed state machine, mandatory ARF node)
- LLM: Nebius Token Factory (Nemotron 120B)
- Enterprise Data: CRAFT MCP (schema discovery, SQL generation)
- Governance: ARF RiskEngine (Bayesian, policy engine)
- Visualizations: Plotly (risk comparison, expected loss, treemap)
- Audit: fpdf2 PDF generation + JSON Lines audit log
- Testing: pytest (9 tests covering blast‑radius, execution boundary, graph structure, audit sanitization)
Challenges we faced
- Integrating three separate systems (CRAFT, Nebius, ARF) into a single seamless demo
- Making Bayesian inference understandable to a non‑technical audience in under 10 seconds
- Ensuring the ARF governance node cannot be bypassed – verified by automated tests
- Building a cinema‑mode auto‑play that works reliably even when the manual “Investigate” button is skipped
- Generating a professional PDF audit trail with proper layout and no encoding issues
Accomplishments we're proud of
- Zero bypass possible – the LangGraph topology enforces ARF governance, verified by tests
- Counterfactual analysis – ARF doesn't just block; it finds the maximum safe action an agent can take
- Full audit trail – every incident produces a downloadable PDF and a structured audit log
- Mathematical rigor – Bayesian formulas are rendered live in the demo and explained in plain language
- Cinematic presentation – the auto‑playing demo tells a complete story without manual clicks
What we learned
- Enterprise AI agents desperately need an independent governance layer – confidence scores are not enough
- Bayesian methods provide a principled way to balance risk and benefit
- Counterfactual reasoning turns a gatekeeper into a partner – ARF shows how to make a plan safe, not just that it's dangerous
- Streamlit + LangGraph can deliver a production‑grade prototype in hours
What's next for ARF Sentinel
- Connect to live CRAFT MCP and Nemotron API (the mock data is clearly marked)
- Add a policy editor so organisations can define their own blast‑radius thresholds
- Integrate with CI/CD pipelines so ARF can govern automated dependency updates in real time
- Expand to other enterprise domains (database schema changes, feature flags, access control)
Log in or sign up for Devpost to join the conversation.