MaryGuardian

A local document assistant that organizes PDFs, spreadsheets, and text files, identifies document types and duplicates, tracks transactions and payment dates, and answers questions about your files.

Comment

Track: Personal AI

Tagline: Private paperwork bodyguard: catches changed bank details

Long tagline: A private, always-on paperwork bodyguard: Nemotron on Nebius Token Factory reads and drafts, local code catches the fraud, and a Guardian gateway makes sure your personal data never leaves unmasked.

Inspiration

Most of us keep a pile of paperwork: invoices, insurance policies, utility bills, contracts with expiry dates. Cloud "AI organizers" want you to upload all of it, and that is a hard sell for tax IDs, IBANs and medical records. But filing is the easy part. The expensive mistakes happen between documents: an invoice that looks like last quarter's but pays a different bank account, a "we have changed our bank details" letter that arrives through the very same channel as the invoice, a PDF with instructions hidden in white text that are written for your AI, not for you. We wanted an assistant that protects you from your paperwork, and never gets a blank check over your data while doing it.

What it does

MaryGuardian is a private, always-on paperwork bodyguard for freelancers, families and small practices. Drop files into a folder and it:

  • watches the folder and indexes new files incrementally (SHA-256, so renames and duplicates are free);
  • reads and sorts them: PDF, Word, Excel, PowerPoint, OpenDocument, text, CSV and XML, with rules that understand English, Russian and Kazakh. 19 document types plus other, with issuer, number, dates, amounts and deadlines. Rules first, then NVIDIA Nemotron when the rules are unsure;
  • tracks deadlines (expired, expiring within 30 days, due);
  • checks the documents against each other, locally and without any model: the payee account against the vendor's history and against accounts you confirmed yourself; "our bank details have changed" notices (a notice from the same channel as the invoice is never treated as confirmation); IBAN checksums; reused invoice numbers with a different amount or account; outlier amounts; impossible dates. Every flag carries its evidence and its explanation, not just a score;
  • shows the evidence on the page itself (X-ray): green for what was found, red for a bank account that differs from the one on file, black for what Guardian hides from the cloud, orange for phrases that look like instructions to an AI, purple for text printed in white that a person cannot see but a model would read;
  • screens for prompt injection before any model sees a document; a suspicious document is kept away from every model;
  • acts for you, with your approval: drafts a verification request to the vendor when the bank details change, drafts letters, records Confirmed by phone in a local registry, answers questions about your library with [id] citations, and learns from your corrections. Nothing is sent or exported without you.

Every model call passes through Guardian, a local gateway: a default-deny action allow-list, PII masking before anything leaves your machine (cards, IBAN, SSN, e-mail, phone numbers, tax IDs, company names and document numbers become placeholders like [IBAN_1]; names and addresses you list are masked the same way), a fail-closed leak check, and an outbound ledger showing exactly what the cloud saw. Only the local app can map placeholders back to real values, and only in memory for the duration of one request. Your files are never modified or deleted.

The demo story: four documents from one vendor. The agreement and an old invoice show account A. A new invoice shows account B, and a letter says "our bank details have changed" and gives account B. An organizer files all four. MaryGuardian flags the new invoice as high risk, says why the letter proves nothing, and tells you to verify by phone using a number from your own records.

How it fits the Personal AI track: it is always on (a folder watcher); private by construction (masking, local-only checks, read-only inbox, audit log); it has persistent memory (the document library, the registry of accounts you confirmed, your corrections and profile, all in local SQLite); it works on the information you choose (the folder you point it at and the terms you protect); and it carries out tasks (drafts, verification requests, exports) behind a human approval step.

How we built it

  • Core: C++17 with MuPDF, SQLite, libcurl and a loopback-only HTTP server with a per-run token and Host/Origin checks. This core existed before the hackathon.
  • NVIDIA Nemotron on Nebius Token Factory, tiered: Nano classifies every new document, Super extracts fields when rules and Nano disagree and writes drafts and verification requests, Ultra answers questions over the library. One OpenAI-compatible API made tiering a configuration change. Models and keys come from environment variables.
  • Models propose, code decides: every value Super returns is validated in code (real ISO dates, amounts that match a number in the text, issuer and numbers that occur verbatim) before it is stored.
  • The checks that matter most use no model. Consistency checks are deterministic and local. The database keeps only a per-install salted SHA-256 fingerprint of each account plus its last four characters. X-ray reads MuPDF's text layer with the colour and position of every glyph, which is how white-on-white lines are found.
  • Built during the hackathon: the remote Nemotron backend with 3-tier routing; the PII-masking gateway with outbound ledger; field validation; multilingual classification rules; consistency and fraud detection; X-ray; correction memory; folder watcher; parallel classification; Linux port; Privacy, Settings and Activity screens; an optional local-model downloader; a synthetic dataset generated by the binary itself; and the test suite. Hosted demo: https://maryguardian.combi.kz/.
  • Caching is central: unchanged files are never re-read, model answers are cached by (model + prompt), and the Activity screen shows calls made, tokens used and tokens saved.

Challenges we ran into

  • Reconciling "your data stays yours" with a cloud model. Our answer is minimal, masked snippets (at most 1,200 characters per document), a fail-closed gateway and a full audit trail. We are explicit about the limits: structured identifiers are detected automatically, while names and addresses must be listed as protected terms.
  • Making a fraud flag trustworthy. A single changed account can be an attack or a legitimate change, and "trust on first use" cannot catch a forged first document. So every flag explains its baseline, a notice from the same sender never counts as confirmation, and only the user's own confirmation clears a flag.
  • Reasoning models spend tokens on thinking. Small token budgets produced empty content with finish_reason: length. We add reasoning headroom and retry with a doubled budget.
  • Multilingual rules in C++. std::regex works on bytes, so \b does not understand Cyrillic. We lower-case UTF-8 while preserving byte length (so match offsets still point into the source) and parse month-name dates by hand.
  • Hidden instructions inside documents, and proving that nothing sensitive reaches the provider. We test against a mock API that records every request body and fail the build if a secret appears in it.
  • Porting a macOS-first C++ app to Linux: hashing, build system, MuPDF packaging, socket handling.

Accomplishments that we're proud of

  • A working end-to-end agent: watch → read → check → explain on the page → draft → approve, with the cloud seeing only masked text.
  • The changed-bank-account scenario works as described above, including Confirmed by phone: the account is recorded, the flag turns into an info note and older invoices stay clean.
  • A planted invoice with prompt-injection text hidden in white is found, kept away from every model, highlighted in X-ray and recorded in the audit log.
  • Tests that check the privacy claims instead of asserting them: the provider's view of every request is free of IBANs, e-mails, SSNs and protected names, and the database never contains a raw account number.
  • Cheap re-scans. In an early test run on 15 mixed documents, hashing the files took 0.03 seconds, 23 Nemotron requests went out with 93 sensitive values replaced by placeholders, and none were blocked.

What we learned

  • The most valuable checks needed no model at all. Models read and draft; code decides what is true and what is risky.
  • Evidence beats scores: a flag that names the document, the account and the baseline gets acted on; a risk number does not.
  • Tiering models (small for bulk, large for reasoning) cuts latency and credits, and doing retrieval in code instead of giving the model tools keeps the agent safer and simpler.
  • Privacy claims need tests, not adjectives.

What's next for MaryGuardian

OCR for standalone images in the default build, legacy .doc/.xls, .eml import so the "notice from the same channel" check can see real e-mail headers, more language packs (DE/FR/ES) with region-specific PII patterns, local NER to catch names automatically, and a self-hosted Nemotron endpoint on Nebius for users who want everything inside their own tenant.

Built with

C++17 · MuPDF · SQLite · libcurl · NVIDIA Nemotron 3 (Nano, Super, Ultra) · Nebius Token Factory · CMake · (optional) llama.cpp with Qwen

Testing instructions

Open https://maryguardian.combi.kz/ (synthetic data only, settings are read-only, state is shared between visitors). If the library is empty, press Scan.

  1. The changed bank account. In Library, open the Needs attention view and open INV-2026-0106_brightwave.pdf. Click Open X-ray: the page replays what was found, the account ending 2051 turns red, and the side panel explains why. Press Cloud view to see what Guardian would hide from the cloud. Press Draft verification request to have Nemotron draft an email through Guardian (nothing is sent). Try Confirmed by phone to see the flag clear, which also changes the shared demo state.
  2. Hidden instructions. Open INV-2026-0999_evil_vendor.pdf from Needs review and click Open X-ray: the two white-on-white lines and the instruction phrases are marked. The Audit log tab records that the document was kept from the model.
  3. What the cloud saw. Open Privacy: open any outbound request to see the masked text, token counts and latency, or paste text containing an IBAN into the masking box.
  4. Deadlines and cleanup. In Library try the Expiring and Expired views, then Cleanup. Activity shows files not re-read, model calls, tokens, and tokens saved by the cache.
  5. Ask. In Ask, try "Which contracts expire this quarter?" and follow the [id] citations.
  6. Memory. Change a document's type and save; Settings → What I learned from your corrections shows the new rule.

To run it yourself, see the README quick start.

Pre-existing project: what was significantly updated

Before the submission period the project was a macOS-first C++ document sorter: incremental indexer, SQLite library, MuPDF and Office text extraction, a loopback server, the Guardian allow-list and audit, a local Qwen classifier and a single-page UI. It had no cloud-model backend, no cross-document checks and no page-level evidence view.

Area Change during the submission period
Models LlmClient abstraction; Nemotron on Nebius Token Factory with Nano/Super/Ultra routing; reasoning-model handling (token headroom, retry, empty content)
Privacy PII-masking gateway (reversible, per request, fail-closed leak check), secrets and strict entity masking, protected terms, hourly request budget, outbound ledger
Extraction Multilingual (EN/RU/KZ) classification rules; escalation to Super for low-confidence documents; every proposed value validated in code
Consistency & fraud detection Payee-account baseline and confirmation registry, "details changed" notices, IBAN checksum, reused numbers, outlier amounts, date checks, risk view
X-ray Page view with evidence boxes, cloud view, hidden-text detection, verification-request drafts behind human approval
Learning Correction memory per company (type and collection)
Always-on Background folder watcher with a "file still being written" guard; parallel classification
Platform Linux port, portable SHA-256, platform layer
UI Privacy, Settings and Activity screens, Needs attention view, X-ray
Data & tests Synthetic dataset generated by the binary (bank-account change, hidden-instruction invoice); unit, integration and leak tests; mock OpenAI server

Details: docs/WHAT_CHANGED.md in the repository.

Built With

Share this project:

Updates

Submission history