LOCALIS

Private local AI platform for document intelligence

Comment

Inspiration

We wanted to build an AI assistant that could understand documents without requiring users to upload their private files to a cloud AI service.

People often have to search through long PDFs, forms, certificates, books, research papers, resumes, invoices, and other documents just to find one specific piece of information. We wanted to make this process faster while keeping the document processing local.

This led to LOCALIS — a private document AI assistant designed around the idea of “Private AI. Running locally.”

What it does

LOCALIS is a local AI document intelligence platform that allows users to upload PDF documents and ask questions about their contents using natural language.

The system:

  • Extracts text from digital PDFs.
  • Uses OCR when a PDF is scanned or does not contain usable text.
  • Organizes extracted content by page.
  • Splits documents into meaningful chunks.
  • Creates embeddings for semantic search.
  • Uses hybrid retrieval to find relevant evidence.
  • Answers questions using the retrieved document context.
  • Shows source pages and chunks so users can verify the answer.
  • Keeps document processing and AI inference local in the current prototype.

LOCALIS is designed to be document-type agnostic. Instead of creating a separate workflow for every type of document, the system uses the document content and the user's question to determine what information is relevant.

How I built it

LOCALIS was built as a modular full-stack application.

Frontend

  • React
  • Vite
  • Modern responsive UI

Backend

  • Python
  • FastAPI

Document Processing

  • PyMuPDF for PDF processing and text extraction
  • RapidOCR for scanned documents
  • Custom text cleaning and page-aware chunking

Retrieval

  • Sentence Transformers
  • all-MiniLM-L6-v2 embeddings
  • Semantic similarity
  • Keyword and evidence-aware retrieval

Local AI

  • Qwen3-0.6B as the local language model
  • A fast evidence extraction path for simple factual questions
  • Local model fallback for more complex questions

The architecture is modular so individual AI components can later be optimized for Snapdragon-powered PCs without redesigning the complete application.

Challenges I ran into

One of the biggest challenges was making PDF extraction reliable across different document types.

Some PDFs contained normal selectable text, while others had unusual text positioning or were scanned documents. In some cases, words could become incorrectly joined during extraction.

To solve this, I implemented multiple extraction strategies, text cleaning, page-aware processing, and OCR fallback.

Another challenge was retrieval accuracy. A document may contain many similar names, numbers, dates, and other pieces of information. Simply returning the most semantically similar chunk was not always enough.

I therefore improved retrieval using a combination of semantic similarity, keyword matching, token coverage, phrase matching, and evidence signals.

Performance was another challenge because the application is designed to run locally. Larger language models were too slow on my development laptop, so I moved to a smaller local model and added a fast document-evidence path for simple questions.

Accomplishments that I'm proud of

I am proud that LOCALIS can process different kinds of PDFs without requiring a separate hard-coded workflow for each document category.

The same system can be used to ask questions about a book, an application form, a certificate, a resume, a research paper, an invoice, or a notice.

I also built the complete pipeline from PDF ingestion to OCR, chunking, embeddings, retrieval, local AI answering, and source display.

Another important accomplishment was designing LOCALIS with local processing in mind rather than treating cloud AI as the default architecture.

What I learned

Building LOCALIS taught me that document AI is not only about choosing a language model.

Reliable document understanding requires a complete pipeline:

Document ingestion → extraction → OCR → cleaning → chunking → retrieval → evidence selection → answer generation

I learned how PDF extraction can fail, why retrieval quality is important before generation, and how smaller local models can be combined with deterministic document processing to improve practical performance.

I also learned how important modular architecture is when building AI applications that may eventually need to run on different hardware platforms.

What's next for LOCALIS

The next step is to optimize LOCALIS for Snapdragon-powered HP PCs.

I plan to evaluate suitable optimized models and runtimes, including Qualcomm AI Hub-compatible approaches where applicable, and benchmark inference speed, memory usage, and overall responsiveness on Snapdragon hardware.

Future versions can also expand beyond document intelligence into:

  • Local voice intelligence
  • Local vision intelligence
  • More advanced document understanding
  • Better multimodal document analysis
  • Hardware-aware model selection
  • More efficient on-device inference

The long-term goal is to make LOCALIS a practical private AI workspace where users can work with their documents and other information directly on their own device.

Share this project:

Updates