Navigation Bar
Before Prompt Model inspection
Latency and PII Tokens Protected
Privacy AI-Gap Log
Main Page After Entering SAYZE_API_TOKEN
Main Page Before Entering SAYZE_API_TOKEN
Prompt Entered, Model Selection (Nemotron-Lightning + Tavily)

Inspiration

Modern AI productivity assistants face a fundamental trade-off: privacy versus capability. To give an assistant access to personal email, bank statements, and schedule coordination, users are forced to send unencrypted, sensitive personal data to cloud endpoints. Conversely, local privacy-first models often lack the reasoning capability required for multi-step agentic execution and real-time web research. We created SAYZE OS to eliminate this trade-off. Our vision was to construct a zero-trust, locally governed "Executive Operations Agent" that acts as a secure air-gap between personal data and external LLMs. By combining local privacy guardrails with cloud-orchestrated reasoning models, SAYZE OS enables automated task execution and web research without compromising sensitive data.

What it does

SAYZE OS is an always-on personal operations engine that coordinates tasks, manages schedules, extracts insights from documents, and conducts real-time web research while protecting private user data. Zero-Trust Guardrail Engine: Automatically detects and redacts Personally Identifiable Information (PII), credit card numbers, financial records, and credentials before prompts leave the local environment. Dynamic Tiered Routing: Routes high-level planning tasks to heavy reasoning models (Nemotron-3-Ultra-550b) on Nebius Token Factory, while delegating low-latency tool execution and formatting to lightweight models (Nemotron-3-5-Lightning). Persistent Memory Graph: Maintains long-term user preferences, habits, and situational context across sessions using vector and graph-based retrieval. Real-Time Grounded Web Intelligence: Integrates the Tavily Search API to ground agent decisions in real-time internet context—such as tracking flight changes or live vendor pricing—without exposing personal details to search queries.

How we built it

SAYZE OS uses a microservices architecture hosted on Nebius AI Cloud and Nebius Token Factory endpoints: Local Security Pipeline (NVIDIA NemoClaw): Implemented an early-stage interception layer using NVIDIA NemoClaw principles to scrub and tokenize PII prior to upstream transmission. Inference & Model Orchestration (Nebius Token Factory): Deployed serverless API endpoints using nebius/nvidia/Nemotron-3-Ultra-550b-a55b for multi-step reasoning, paired with nebius/nvidia/Nemotron-3_5-Lightning for fast function-calling and schema parsing. Agent & Memory Framework: Built on top of the Hermes Agent Framework to handle persistent episodic memory using Qdrant vector storage. Live Search Grounding: Embedded the Tavily API directly into Nemotron’s tool execution loop for live web updates.

Challenges we ran into

Guardrail Latency Penalty: Intercepting every prompt with local embedding checks and PII regex parsing introduced an initial delay. We mitigated this by parallelizing the regex sanitization pipeline using async worker threads in FastAPI. Context Preservation During Masking: Simple string replacement can strip critical context. For example, replacing a monetary figure can prevent the model from performing arithmetic. We resolved this by implementing semantic pseudonyms, replacing sensitive variables while retaining their data type. Function Call Schema Drift: Aligning Tavily search responses with strict JSON tool schemas required standardizing output parsing rules on the Nemotron-3-5-Lightning endpoint to avoid model hallucinations during execution loops.

Accomplishments that we're proud of

Sub-Second Guardrail Processing: Achieved a fast local PII sanitization pass before routing payloads to external Nebius endpoints. Optimal Cost-Efficiency Ratio: Successfully engineered a dual-tier model router on Nebius Token Factory. By routing routine formatting tasks away from large reasoning models, we achieved significant compute savings while preserving top-tier accuracy. Zero PII Leakage: Validated that sensitive personal data remains within the local boundary during live web research calls via Tavily.

What we learned

Routing Matters More Than Model Size: Intelligent payload routing between Nemotron-3 Ultra and Nemotron Lightning yields better system latency than relying on a single monolith model. Deterministic Security is Essential: LLMs cannot be trusted to redact their own input prompts. Privacy guardrails must run deterministically before prompt ingestion. Nebius Serverless Speed: Serving open NVIDIA Nemotron weights via Nebius Token Factory provided noticeably lower time-to-first-token (TTFT) compared to standard self-hosted container deployments.

What's next for SAYZE OS

On-Device Local Model Guardrails: Porting the initial NemoClaw filtering stage to run natively on NVIDIA Jetson edge devices for complete local isolation. Multi-Modal Document Processing: Expanding Nemotron-3.5 Vision integration to process financial receipts, physical mail scans, and complex PDF reports natively. Encrypted Graph Memory: Introducing end-to-end client-side encryption for the Hermes long-term memory store, ensuring memory vectors remain encrypted at rest and in transit.