-
-
Landing Page
-
Dashboard Overview
-
Activity
-
Agent
-
Document Ingestion
-
Knowledge Graph
-
Memory
💡 Inspiration
Every AI assistant I used had the same three problems:
- It forgot everything. Each conversation started from zero, so I kept re-explaining my projects, decisions, and preferences.
- It made things up. When it didn't know something, it answered confidently anyway, with no way to check where the answer came from.
- It acted without asking. Agent frameworks were racing toward full autonomy, letting LLMs create files, call APIs, and run code with little oversight.
I wanted an assistant that behaves more like a trustworthy colleague: one that remembers what you've taught it, reasons only from evidence, and asks before doing anything consequential. That became SynapseOS, a privacy-first personal AI operating system.
🛠️ How We Built It
SynapseOS is a full-stack system with three core layers: Memory → Reasoning → Action.
1. Persistent Multi-Modal Memory
Notes, documents (.md, .txt, .pdf, .json, .csv), decisions, and interactions are chunked and stored in three representations:
- Relational records for structured CRUD and provenance
- Vector embeddings for semantic search
- A knowledge graph linking entities, memories, and documents
Every query, vector search, and graph traversal is scoped to the authenticated user's workspace ID, so memories from different users or workspaces never mix.
2. Evidence-Grounded RAG with an Abstention Guard
For each question $q$, we retrieve candidate chunks $d_i$ and score them by cosine similarity:
$$ \text{sim}(q, d_i) = \frac{\mathbf{q} \cdot \mathbf{d}_i}{|\mathbf{q}| \, |\mathbf{d}_i|} $$
If the best match falls below a grounding threshold $\tau$:
$$ \max_i \; \text{sim}(q, d_i) < \tau \;\Rightarrow\; \text{abstain} $$
the assistant explicitly says it doesn't have enough evidence instead of fabricating an answer. When it does answer, every claim comes with citations to the source chunks, so users can verify it.
3. Bounded Agent Runtime with Human Approval Gates
The planner, powered by NVIDIA Nemotron via Nebius Token Factory, breaks a user goal into a structured plan with typed arguments and predicted impact. Each step is classified:
- Safe tools (
search_knowledge,retrieve_context) run autonomously. - Consequential tools (
create_project,modify_filesystem, code execution) pause the pipeline and create an approval ticket.
Approval tickets expire, are single-use, and are bound to the session owner, so one user can't approve another user's action and a ticket can't be replayed. After execution, a Reflection Engine turns outcomes into new observational memories. It can never escalate its own privileges.
4. Security Hardening
- SSRF guard: agent web requests are validated against loopback, cloud metadata (
169.254.169.254), and private subnets (10.0.0.0/8,172.16.0.0/12,192.168.0.0/16). - PII and secret redaction: API keys, tokens, emails, and phone numbers are scrubbed before any prompt leaves for an external model provider.
- Auth: Argon2id password hashing, JWT bearer tokens, and token invalidation on logout.
5. Stack
- Frontend: Next.js 14, React 18, TypeScript, Tailwind, React Query, Zustand (deployed on Vercel)
- Backend: FastAPI, Python 3.12, Pydantic v2, HTTPX (deployed on Render)
- Inference: Nebius Token Factory serving NVIDIA Nemotron, with Ollama support for fully local runs
- Storage: SQLite (embedded mode) with a Docker Compose stack for PostgreSQL 16, Qdrant, Neo4j, and Redis
- Quality: Pytest, live E2E acceptance tests, full-stack contract tests, and a custom RAG/agent evaluation harness
📊 Results
We built an automated evaluation harness so our claims are measurable, not just asserted:
| Metric | Result |
|---|---|
| Semantic Retrieval Recall@K | 100% (6/6) |
| Grounded Answer & Citation Rate | 100% (6/6) |
| Abstention on Unsupported Questions | 100% (3/3) |
| Consequential Action Approval Gate | 100% (1/1) |
| Unauthorized Approval Rejection | 100% (1/1) |
| Backend Test Suite | 52 passed, 1 skipped |
| Live E2E Acceptance | 8/8 passed |
| Frontend/Backend Contract Tests | 12/12 passed |
These are early-stage evaluation sets. Expanding them to hundreds of adversarial cases is a top priority.
🧗 Challenges We Faced
- Calibrating abstention. If $\tau$ is too high, the assistant refuses questions it could answer. If it's too low, it hallucinates. Finding the balance took repeated tuning against both supported and deliberately unsupported queries.
- Designing an approval gate that's actually secure. A "Confirm" button is easy. Preventing replay, cross-user approval, and stale tickets required expiry, single-use consumption, and ownership checks, each with its own tests.
- Keeping three memory representations consistent. Deleting a memory has to remove it from the relational, vector, and graph stores together, or "forgotten" data lingers.
- SSRF in agent tools. Letting an agent fetch URLs opens the door to probing internal networks and cloud metadata endpoints. We had to validate resolved IPs, not just hostnames.
- Free-tier deployment constraints. Cold starts and ephemeral storage on free hosting pushed us to build health and readiness probes and a deterministic offline mode for testing without cloud dependencies.
🏆 Accomplishments We're Proud Of
- A working memory → plan → approve → act → reflect loop running live end to end
- Human-in-the-loop safety that is enforced by the backend, not just shown in the UI
- A test and benchmark suite that rarely makes it into hackathon projects
- A clean separation between safe autonomy and consequential actions that scales to new tools
📚 What We Learned
- Trust is a feature. Citations, abstention, and approval gates made the assistant more useful, because users could rely on it.
- Large models like Nemotron are excellent structured planners when given typed tool schemas, which makes plans easy to validate before execution.
- Security has to be designed in from the start. Workspace isolation and SSRF protection would have been painful to add later.
- Measure everything. Building the evaluation harness early caught regressions that manual testing missed.
🚀 What's Next
- Larger adversarial evaluation sets (prompt injection, cross-workspace leakage, multi-hop retrieval)
- A desktop app (Electron) for fully local, on-device memory
- Fine-grained, per-tool permission policies users can configure
- Memory decay and consolidation, so the knowledge base stays relevant over time
- Team workspaces with role-based approval chains
Log in or sign up for Devpost to join the conversation.