Receipts — Sovereign Memory Layer

Receipts: Local-first AI memory with auditable receipts — source, confidence & decay for every fact you can inspect, correct or delete.

Comment

Receipts — Sovereign Memory Layer for AI "Never trust an AI's memory without asking for the receipt." Verifiable memory layer for personal AI agents with cryptographic provenance, parameterized half-life decay, live NVIDIA Nemotron inference, Tavily web verification, and encrypted cross-device sync.

Inspiration Modern AI assistants remember personal preferences, project context, and factual statements inside opaque vector databases or proprietary cloud silos. When an assistant makes an assumption, users have no way to answer fundamental questions: Where did this memory come from? When was it recorded, and is it already stale? How can I edit, correct, or permanently erase it? How do I prevent the model from quietly hallucinating claims about me? Receipts introduces the concept of Sovereign Memory. Every remembered detail is treated as an immutable, cryptographically hashed Memory Receipt that you own locally. The model is strictly prohibited from asserting unverified facts: every response must explicitly cite the receipt ID backing its claim.

What It Does Local-First Sovereign Vault (SQLite + SHA-256 Hash Chain): Stores memories locally with exact provenance excerpts, timestamps, confidence scores, and cryptographic hash chaining back to a genesis block (GENESIS). Tampering is immediately flagged via /api/integrity. Grounded Live Inference with NVIDIA Nemotron: Synthesizes answers using only verified receipts, citing exact bracketed receipt IDs (e.g. [Receipt #REC-AUS7]) with zero hallucinations. Autonomous Fact Extraction (Nemotron Nano): Parses ongoing conversations in real time to detect newly stated facts, automatically proposing structured receipts for one-click approval. Interactive Memory Graph with Organic Physics: Visualizes memories as an interactive knowledge graph with smooth celestial floating physics, dynamic repulsion, edge spring tension, real-time pause/resume layout freezing, and tag visibility toggles. Parametric Half-Life Decay & Proactive Maintenance: Replaces arbitrary TTLs with cognitive decay curves (Lifestyle: 90d, Projects: 21d, Ephemeral: 7d). A proactive Copilot flags stale memories with one-click Reaffirm, Edit, or Forget actions. Tavily Web Search Ingestion & Source Verification: Ingests live web research and continuously validates source URLs against link rot and content drift using automated term-overlap checks. Authenticated Cross-Device Sync (AES-256-GCM): Enables peer-to-peer sync across devices without central cloud lock-in via PBKDF2 passphrase encryption. Outbound Data Privacy Ledger: Audit logs every outbound prompt, model ID, latency, and token count sent to external APIs, with authorization headers cryptographically redacted.

How We Built It Backend: Python 3.11+, FastAPI, SQLite with PRAGMA secure_delete = ON (overwrites freed database pages with zeros so deleted memories cannot be recovered from disk slack space). AI & Inference: NVIDIA Nemotron open models via OpenAI-compatible endpoints with bounded retries, streaming support, and structured JSON parsing. Web Intelligence: Tavily API for real-time web search ingestion and automated source verification. Frontend: Zero-dependency vanilla ES6 JavaScript, HTML5 SVG canvas, and customized Thermal Terminal CSS design system. Cryptography: Cryptographic SHA-256 hash chaining for tamper detection, AES-256-GCM authenticated encryption for sync bundles.

Challenges We Overcame Strict Citation Enforcement: Engineered prompt boundaries ensuring Nemotron grounds every assertion with bracketed receipt tags ([Receipt #...]). True Sovereign Deletion: Enforced PRAGMA secure_delete = ON and tombstone generation to guarantee zero residual data recovery upon sovereign memory erase. Robust Provider Architecture: Handled differences across API endpoints with automated provider resolution and candidate model fallback, ensuring zero crashes. Fluid Physics & Performance: Built custom spring-and-orbit physics for SVG nodes without external libraries like D3, achieving smooth 60 FPS animation with instant freeze-frame pausing. Accomplishments We're Proud Of 100% Live Inference Verified: End-to-end verified inference using NVIDIA Nemotron producing grounded answers with exact bracketed citations in ~1.1 seconds. 100% Automated Test Pass Rate: Clean pytest compliance suite verifying cryptographic tampering detection, secret masking, retry/fallback behavior, and encrypted bundle round-trips. Live Web Intelligence: Verified live Tavily search ingestion and automated term-overlap content verification. Zero External JS Dependencies: The entire UI runs without React, D3, or heavy build tooling.

Accomplishments that we're proud of Zero-Hallucination Grounded Inference with Live Nemotron: Successfully verified end-to-end inference using NVIDIA open models (nemotron-3.5-bolt-e4b and nemotron-3-nano-omni-30b-a3b-reasoning), forcing the model to cite exact bracketed receipt IDs (e.g. [Receipt #REC-AUS7]) for every claim it makes in ~1.1 seconds. Tamper-Evident SHA-256 Hash Chain: Engineered a cryptographic ledger where every memory modification updates a parent hash back to a genesis block (GENESIS), making any silent database edits or hallucinations programmatically detectable via /api/integrity. Zero Residual Storage on Deletion: Enforced PRAGMA secure_delete = ON within SQLite to physically overwrite deleted memory pages with zeros on disk, guaranteeing true sovereign erasure without forensic recoverable slack space. Interactive Physics Graph with Zero Dependencies: Implemented a real-time, 60 FPS celestial spring-and-orbit physics layout in SVG using pure vanilla ES6 JavaScript—complete with live pause/resume layout freezing and dynamic label toggles without React, D3, or heavy node bundles. Live Web Sourcing & Verification: Seamlessly integrated Tavily Web Intelligence to ingest live research and run automated term-overlap drift checks against source URLs. 100% Automated Test Suite: Built a comprehensive offline test and compliance harness verifying secret masking, cryptographic integrity, failure retries, and authenticated sync round-trips with a 100% pass rate.

What we learned Prompt Engineering for Strict Verifiable Provenance: We learned that instructing an LLM to "only use provided context" is insufficient to prevent extrapolation. Enforcing a strict citation syntax ([Receipt #ID]) combined with small, minimal-context retrieval windows was key to eliminating hallucinations. Reasoning Models Require Dedicated Token Budgets: Working with reasoning models like Nemotron Nano Omni taught us that internal reasoning tokens (reasoning_content) consume generation limits before final answers are emitted. Designing robust fallback parsing for reasoning streams is essential for production agents. Local-First Privacy vs. Cloud Inference Trade-Offs: Designing the Outbound Data Privacy Ledger highlighted the tension between personal privacy and cloud intelligence. Masking authorization headers, logging outbound prompt payloads locally, and cryptographically auditing transfers gives users visibility into what leaves their machine. Physics Simulations on the DOM: Managing velocity damping and spring equilibrium across dynamically filtered SVG nodes without third-party libraries provided deep insights into browser animation frames and render performance.

What's next for Receipts — Sovereign Memory Layer 100% Offline Air-Gapped Local NIM: Package quantized Nemotron Nano directly in Docker via local NVIDIA Container Toolkit, enabling true zero-cloud, 100% air-gapped sovereign memory execution. Multimodal Visual Receipts (Nemotron Parse): Extend fact intake to analyze physical grocery receipts, contracts, whiteboard sketches, and handwritten notes using Nemotron OCR and vision-language capabilities. Hardware Sovereign Enclave Attestation: Bind the root memory hash chain into hardware security modules (TPMs or YubiKeys) so that memory state changes require cryptographic physical attestation. Decentralized Multi-Agent Exchange: Allow personal agents to exchange verified memory receipts peer-to-peer (e.g., sharing a dietary preference or scheduling calendar) using zero-knowledge proofs without exposing full vault histories.

Built With

Share this project:

Updates

Submission history