VaultVerifyFinal

Privacy-first compliance automation. VaultVerify uses local RAG and re-ranking to draft grounded, cited answers from your private documents. Speed of AI, security of a vault.

Comment

🛡 VaultVerify

Evidence-Backed Private Questionnaire Assistant

ThunderHacks 2026 — Shield Identity Platinum Challenge

Answer sensitive compliance questionnaires using your own private documents.
Every answer is grounded in your uploaded files. Nothing ever leaves your machine.

What It Does

Organizations waste hours manually searching through policies, architecture docs, and audit reports to answer compliance questionnaires (SOC 2, ISO 27001, vendor risk, etc.). VaultVerify solves this with a guided 4-screen workflow:

  1. Intake — Enter your org context, upload private documents, define your questions
  2. Review Console — AI drafts answers from your documents; you Approve, request Changes, or Reject each one
  3. Unresolved Queue — Rejected items are surfaced again for a second pass before you can export
  4. Response Package — Download a clean PDF or CSV of all approved answers with source citations

Project Structure

ShieldIdentity/
│
├── review_console.html   ← Main UI — open this in your browser
├── chat.html             ← Optional free-chat interface for exploring documents
│
├── api.py                ← FastAPI server (connects UI ↔ backend)
├── ingest.py             ← Indexes uploaded documents into ChromaDB
├── retriever.py          ← Semantic search + cross-encoder re-ranking
├── generator.py          ← RAG answer generation via local Ollama LLM
├── exporter.py           ← Generates PDF + CSV response packages
│
├── .env                  ← Configuration (model name, paths)
├── .env.example          ← Template — copy to .env
├── .gitignore            ← Keeps .env and DB out of version control
├── requirements.txt      ← Python dependencies
│
├── documents/            ← Drop your PDFs/DOCX here before indexing
├── vaultverify_db/       ← ChromaDB vector store (auto-created)
└── uploads/              ← Files uploaded via the UI (auto-created)

How It Works

Your Documents (PDF / DOCX / PNG)
         │
         ▼
    ingest.py          Extracts text, chunks with overlap,
                       embeds with all-MiniLM-L6-v2,
                       stores in local ChromaDB
         │
         ▼
   retriever.py        Stage 1: Cosine similarity search (top 20 chunks)
                       Stage 2: Cross-encoder re-ranking (top 5)
                       Returns: doc name, page, section, excerpt, confidence
         │
         ▼
   generator.py        Builds a strict RAG prompt with evidence chunks
                       Calls Ollama (local LLM — no internet required)
                       Returns: grounded answer + citations
         │
         ▼
   review_console.html Approve / Needs Changes / Reject each answer
                       Feedback triggers AI revision
                       Rejected items queued for rework
                       Export approved answers as PDF or CSV

Privacy guarantee: All LLM inference runs through Ollama on your local machine. ChromaDB stores vectors on local disk. No data is sent to OpenAI, Anthropic, or any external service.

Setup Guide

Prerequisites

Tool Purpose Install
Python 3.10+ Backend runtime python.org
Ollama Local LLM inference ollama.com/download
Tesseract (optional) OCR for image files GitHub

Step 1 — Install Ollama and pull a model

# After installing Ollama, pull the recommended model (~2GB)
ollama pull llama3.2

# Verify it downloaded
ollama list

Ollama starts automatically on Windows. On Mac/Linux you may need:

ollama serve

Step 2 — Install Python dependencies

pip install -r requirements.txt

If pip isn't found, try:

python -m pip install -r requirements.txt

Step 3 — Configure environment

The .env file is pre-configured with sensible defaults. Edit it if you want a different model:

OLLAMA_BASE_URL=http://localhost:11434
OLLAMA_MODEL=llama3.2
EMBED_MODEL=all-MiniLM-L6-v2
MAX_TOKENS=1024
CHROMA_DIR=./vaultverify_db
UPLOAD_DIR=./uploads

No API keys needed.

Step 4 — Add documents and index them

# Create the documents folder and add your files
mkdir documents
# Copy PDFs, DOCX files, or images into ./documents/

# Index everything
python ingest.py ./documents

You should see output like:

VaultVerify Ingestor — 3 file(s) found
✓ 847 chunks indexed from 47 pages
Done. Vector store contains 847 chunks total.

Step 5 — Start the backend server

python -m uvicorn api:app --port 8000 --reload

Keep this running in a terminal throughout your session.

Step 6 — Open the UI

Double-click review_console.html in File Explorer, or open it in your browser:

file:///C:/path/to/ShieldIdentity/review_console.html

For the free-chat interface:

file:///C:/path/to/ShieldIdentity/chat.html

Full Startup Checklist

[ ] ollama pull llama3.2          (one-time, ~2GB download)
[ ] pip install -r requirements.txt
[ ] python ingest.py ./documents  (after adding files to ./documents/)
[ ] python -m uvicorn api:app --port 8000 --reload
[ ] Open review_console.html in browser

API Reference

The FastAPI server exposes these endpoints (interactive docs at http://localhost:8000/docs):

Method Endpoint Description
GET /health Check Ollama status, model availability, chunk count
GET /documents List all indexed documents
POST /documents/upload Upload and index a new document
POST /evidence/retrieve Get ranked evidence for a question
POST /answers/generate Generate a grounded answer
POST /answers/rework Revise an answer with reviewer feedback
PATCH /answers/{id}/status Set status: approved / rejected / changes
GET /answers List all answers in session
POST /export Generate PDF and/or CSV response package
GET /export/download/{filename} Download an exported file

Recommended Models

Model Size Notes
llama3.2 ~2GB Recommended — fast, good for demos
mistral ~4GB Better quality, slightly slower
llama3.1:8b ~5GB Best answer quality, needs 8GB+ RAM
phi3:mini ~2GB Good fallback for low-RAM machines

To switch models, edit .env:

OLLAMA_MODEL=mistral

Troubleshooting

"Ollama not reachable"

# Start Ollama manually
ollama serve

"Model not available"

ollama pull llama3.2
ollama list   # verify it appears

"uvicorn not recognized"

python -m uvicorn api:app --port 8000 --reload

"No supported files found" when ingesting

  • Make sure files are inside ./documents/
  • Supported formats: .pdf, .docx, .png, .jpg, .jpeg
  • Check with: dir documents (Windows) or ls documents (Mac/Linux)

"Insufficient evidence found" when asking questions

  • Documents may not be indexed yet — run python ingest.py ./documents
  • Try a more specific question that matches keywords in your documents
  • Check chunk count at http://localhost:8000/health

Port 8000 already in use

python -m uvicorn api:app --port 8001 --reload
# Then update the API constant in both HTML files from 8000 to 8001

Shield Identity Challenge — Capability Pillars

Requirement Implementation
Private Knowledge Intake File upload UI → FastAPI → ChromaDB local vector store
Evidence-Backed Answering Two-stage retrieval (dense search + cross-encoder reranking), source cards with page + section + excerpt
Human Approval Loop Approve / Needs Changes / Reject buttons; feedback textarea triggers AI revision
Rework & Final Deliverable Unresolved queue screen; PDF export via ReportLab; CSV export client-side

Tech Stack

Layer Technology
LLM Inference Ollama (local) — llama3.2 / mistral
Embeddings sentence-transformers / all-MiniLM-L6-v2
Re-ranking cross-encoder/ms-marco-MiniLM-L-6-v2
Vector Store ChromaDB (local persistent)
PDF Parsing PyMuPDF (fitz)
DOCX Parsing python-docx
OCR Tesseract + Pillow
Backend FastAPI + Uvicorn
PDF Export ReportLab
Frontend Vanilla HTML/CSS/JS (no framework, no build step)

Built for ThunderHacks 2026 · Shield Identity Platinum Challenge · thunderhacks.algomau.ca

Built With

Share this project:

Updates