Inspiration

We are entering the era of ambient computing, where smart home devices—such as Alexa+, FireTV, Ring 3D radar, and AI wearables—are evolving from passive information assistants into autonomous agents with purchasing power.

However, giving autonomous LLMs direct access to credit cards introduces critical vulnerabilities:

Runaway LLM Hallucinations: An agent caught in an automated loop or misunderstanding quantities can trigger unauthorized, rapid transactions.
Adversarial Audio & TV Injections: Background audio from a TV commercial or streaming video can trick smart speakers into placing orders without homeowner knowledge.
Absence of Physical Presence Verification: Voice alone does not prove the homeowner is physically in the room, creating huge liability when kids, guests, or rogue scripts are present.

We asked ourselves: How do we unlock the magic of autonomous ambient commerce while making accidental or rogue spending mathematically impossible?

This inspired Fides (named after the Roman goddess of trust): a sovereign, air-gapped policy and authorization gateway that validates physical presence tokens, enforces strict spending mandates, and cryptographically signs every transaction before a single cent leaves your wallet.

What it does

Fides acts as an authoritative, real-time security gateway between ambient AI agents and payment processors:

7-Gate Deterministic Policy Engine: Evaluates every agentic intent in under $15\text{ ms}$ against velocity rules, price ceilings, merchant categories, and duplicate order windows.
Physical Spatial Presence Verification: Correlates Ring 3D radar telemetry and wearable biometric tokens. If an order exceeds budget limits while the house is empty, Fides automatically halts execution and triggers a step-up approval.
Zero Wallet Leak Architecture: AI agents never touch raw payment credentials, bank tokens, or API keys. They only receive time-bound, cryptographically signed approval tokens.
Tamper-Evident SHA-256 Audit Chain: Every decision—approved, escalated, or blocked—is recorded in a cryptographic hash sequence: $$H_i = \text{SHA-256}\Big(H_{i-1} \parallel \text{Payload}_i \parallel \text{Timestamp}_i \parallel \text{Decision}_i\Big)$$ Retroactive modification of any past transaction instantly breaks the chain integrity $\Delta H \neq 0$.
Real-Time Step-Up Escalation: When purchases fall outside baseline policy mandates, Fides pauses the transaction flow and routes an SMS/webhook approval request to the user's phone.

How we built it

Backend & Policy Gateway: Built with FastAPI (Python 3.11) for high-concurrency, asynchronous evaluation. We used Pydantic v2 for rigid data contracts and dynamic YAML policies for living-room spend profiles.
Model Context Protocol (MCP): Implemented an Alexa+ MCP Server (spec version 2025-11-25) using streaming JSON-RPC over HTTP, enabling voice agents to call Fides policy tools natively.
Interactive HUD & Playground: Developed a modern single-screen dashboard in React and Vite, featuring live scenario injection, an Alexa+ voice simulator, decision tree inspection, and an interactive cryptographic ledger viewer.
3D Visual Engine: Implemented hardware-accelerated CSS 3D perspective transforms to render a continuous helical ribbon of live smart device feeds orbiting the trust monolith.
Testing & Media Pipeline: Leveraged headless Playwright and FFmpeg to record 1080p 60fps walkthrough videos and validate sub-$20\text{ ms}$ response latencies across test suites.

Challenges we ran into

Sub-20ms Latency Budget: Voice commerce demands immediate feedback; any delay over $200\text{ ms}$ causes audible pauses. We optimized the policy evaluator to execute all 7 security gates in $t_{\text{eval}} < 12\text{ ms}$.
Concurrency in Hash-Chained Ledgers: When multiple ambient devices submit simultaneous transactions, race conditions can corrupt sequential block linkage. We implemented atomic queue locking to guarantee strict linear ordering: $$H_0 \longrightarrow H_1 \longrightarrow H_2 \longrightarrow \dots \longrightarrow H_n$$
Zero-Trust LLM Defense: We treated all LLM outputs as untrusted input, ensuring prompt injections cannot override mathematical budget rules or presence verifications.

Accomplishments that we're proud of

Zero Wallet Leak Guarantee: Proved that voice assistants can conduct seamless autonomous commerce without ever storing or reading credit card numbers.
100% Deterministic Security: Zero reliance on stochastic LLM judgments for safety; all 7 security checks are backed by verifiable code and cryptographic math.
Native MCP Compliance: Built a production-ready MCP tool suite that works out of the box with modern agent protocols.
Flawless End-to-End Suite: All unit, integration, and concurrency race tests pass cleanly with full audit ledger verification.

What we learned

Ambient commerce is a governance problem, not an AI problem. Large language models are already capable of choosing products; what was missing was the institutional security layer to authorize financial movement safely.
Physical presence is the strongest ambient factor. Combining ambient radar signals with policy evaluation eliminates the risk of accidental voice-triggered purchases.
Standardized protocols win. Aligning with the Model Context Protocol (MCP) enables Fides to protect any agent runtime—from Alexa+ to FireTV, Claude, and local home robotics.

What's next for fides

Hardware Security Module (HSM) Integration: Moving cryptographic signing keys directly onto edge secure silicon (Apple Secure Enclave, AWS Nitro Enclaves).
Zero-Knowledge Proofs (ZKP): Allowing agents to prove authorized purchasing power to merchants without disclosing account balances or identity.
Standardized Open Protocols: Contributing to emerging agentic payment standards (AP2 and x402) so that any smart device maker can integrate Fides guardrails natively.