Nexus

• 

  The Roadmap of the project

• 

  Background phot

About Nexus

Inspiration Every AI assistant today ships your data to someone else's server. We wanted a brain that lives on your machine — one that remembers what you worked on, builds its own automations, and only reaches for the cloud when you explicitly approve it. The name Nexus comes from the idea of a single private hub connecting your conversations, your documents, and your recurring tasks.

What we learned

 Privacy-first design isn't a feature you bolt on — it has to be the architecture. Every subsystem had to ask "what leaves the device?" before being wired in.
 A local 4B-parameter model is genuinely useful for chat and skill generation, but only when you feed it memory context. Without retrieval, even a capable local model feels amnesiac.
 Sandboxing generated code is harder than it looks. AST allow-lists lull you into false security — the moment you permit OS, you've lost. We had to rethink it as a deny-list plus process-level isolation.
 SQLite and ChromaDB have very different threading constraints. Pretending they don't bites you the first time you serve HTTP on a different thread than the engine.

How we built it

 Local LLM: llama.cpp running NVIDIA's Llama-3.1-Nemotron-Nano-4B GGUF, with CPU fallback when no GPU is present.
 Memory: a dual-store design — SQLite for episodic events (timestamped timeline of chats and skill runs) and ChromaDB for semantic retrieval over ingested documents, both embedded with all-MiniLM-L6-v2.
 Skills: natural-language requests go to the local LLM, which returns a JSON spec. The generated code is AST-validated, then run in a subprocess sandbox with a 30-second timeout and resource caps.
 Cloud burst: a privacy gate intercepts complex requests, surfaces an approval modal with the token estimate and reason, and only then forwards to Nebius Nemotron-3-Ultra. Every decision — approved or denied — is written to an append-only audit log.
 Interfaces: a PyQt6 desktop app for daily use, plus a headless HTTP server (single-page UI + JSON API) for SSH and container scenarios. Both share the same engine, so memory, skills, and the privacy gate behave identically.

Challenges

 The privacy gate has no clean "ask the user" path across the desktop/headless boundary. We settled on an injectable approval callback so each host can wire its own UI.
 Keeping the engine single-threaded while the HTTP server stays responsive meant moving the LLM onto a worker thread with a request queue.
 Generated skills kept producing invalid JSON or code with banned imports; we added a retry loop and pre-registration validation to keep the registry clean.
 ChromaDB tries to download its own embedding model by default — we had to compute embeddings ourselves and pass them in explicitly to keep the store fully offline.

Built With

• bash
• cli
• markdown
• python