Palace

A private personal AI with a real memory: it tells you what you decided, whether it changed, and which memories it used. NVIDIA Nemotron on Nebius Token Factory.

Comment

Inspiration

"Are we still hiring a coordinator this fall?" A tired person answers that from memory, and memory is stale: the decision was reversed three weeks later. Palace answers from the record instead. It tells you the current decision, the history behind it, and exactly which notes it used.

Palace is TechEMPOWER's open-source, private-by-default memory assistant. It's the kind of personal AI our digital navigators can set up for someone who can't afford a cloud subscription, or doesn't want their life on someone else's server. It grew out of a memory system our executive director uses every day, where every working conversation is filed on a server in his house. That system was tied to closed, hosted models. We wanted the whole loop on open models.

What it does

Ask what you decided, what you promised, or what happened last time. Palace recalls the relevant memories, sends only those snippets to NVIDIA Nemotron on Nebius Token Factory, and answers with each claim cited by memory id and date.

  • Current decision first. "What did I decide about hiring a coordinator, and did it change?" gets: you decided to hire in July [demo-013]; you reversed it on August 5 because donations came in 40% under plan [demo-016]. That answer came live from Nemotron 3 Ultra in 1.2 s, for under a tenth of a cent.
  • Right-sized models. Short lookups go to Nemotron 3 Nano, ordinary questions to Super, and "did it change?", planning and comparisons to Ultra. The tier and the reason are shown on every answer.
  • "What leaves the house." A panel beside each answer shows the memories recalled, how many characters were sent, the model id, and the exact messages sent to Token Factory. Your archive stays put; only the few snippets for that one answer leave, capped at 600 characters each.
  • Clickable citations. Each [id, date] highlights the memory it came from.
  • It declines when it doesn't know. Both unanswerable questions in our eval were declined, not invented.
  • A safe public demo. It runs on a synthetic archive for a fictional community tech hub. The code refuses to connect to anything that looks like a real home server.

How we built it

  • Models: NVIDIA Nemotron 3 Nano (30B-A3B), Super (120B-A12B) and Ultra (550B-A55B) on Nebius Token Factory, through its OpenAI-compatible API. The client is Python standard library only. palace models checks the three ids against the live catalog before every demo.
  • Router: transparent rules pick the tier and return a human-readable reason.
  • Memory: an HTTP client for palace-daemon (a PostgreSQL + pgvector + BM25 memory store). With no daemon configured, an offline BM25 index over the synthetic archive stands in, so anyone can run it with no server.
  • Grounding: the prompt holds only the numbered memories for the question, and tells the model to cite every claim and to say so when the memories don't answer it.
  • Web chat: one static page, no JavaScript dependencies, light and dark themes, keyboard accessible.
  • Evals: 26 questions with expected memories and tiers, run live on Token Factory: recall@8 0.958, citation validity 1.0, expected-citation 0.958, abstention 1.0. A full run costs about one cent.
  • Tests: 57 offline tests, including the real-palace guard, the router tiers, the citation format, and the demo's hourly and daily caps on live calls.

Challenges we ran into

  • Citation formats differ by model. Nemotron 3 Super sometimes cited as 【demo-008】 instead of [demo-008]. Our scorer saw no citations on those answers, so a "no invented ids" result was vacuous until we normalised both forms.
  • Routing without paying Ultra prices for everything. The keyword router still picks the wrong tier on 7 of 26 eval questions (see Limitations).
  • Demoing personal memory without a real person's memories. We built a synthetic archive with realistic texture (a reversed decision, dated promises) and a code-level guard against real palaces.

Accomplishments that we're proud of

The egress panel. It turns "your data stays private" from a policy promise into something you can check on every answer.

What we learned

  • Ultra costs about 6× Super per answer here ($0.0013 against $0.00022, with Nano at $0.00007), but median latency is about the same (1.3–1.8 s). The cost of a wrong route is money, not time.
  • An eval that can't see a failure reports a perfect score. We now make sure each metric can go red before we trust it.

Limitations

  • The router picks the wrong tier about a quarter of the time (tier accuracy 0.731), in both directions.
  • Recall has no stemming: "hotspots returned" misses a memory that says "Hotspot returns".
  • Palace is only as good as the archive. If a reversal was never written down, it reports the old decision. It doesn't invent one.
  • The real-palace guard is a heuristic, and it can also refuse a legitimate LAN daemon until that host is allowed.

What's next for Palace

A one-command installer so a TechEMPOWER navigator can set up a palace for a neighbor; stemming and a learned router; Hermes Agent tools (recall, remember, brief_me) over the same turn loop; voice through a local speech stack.

How we used AI tools

We wrote much of the code with an AI coding assistant (Claude Code), reviewed and tested by us. The runtime models are NVIDIA Nemotron on Nebius Token Factory.

Built With

Share this project:

Updates

Submission history