home
search result
ai answer

Prism Library — AI answers that cite the book, or admit it doesn't know

Inspiration

As a medical student, my textbooks are thousands of pages, and the answer I need is almost always buried in a single paragraph. Like everyone in 2026, my first instinct was to ask an LLM. It was fast, confident — and occasionally wrong in ways that matter: inventing a drug dosage or misstating a mechanism with total conviction. I realized the problem was never that AI lacks knowledge. It was that AI has no accountability. When a model answers from memory, you cannot tell where knowing ends and imagining begins.

Prism Library began with one rule: an AI that helps you study medicine should never be allowed to hallucinate about medicine. Every answer it gives must point back to a real passage in a real textbook — or it must say, plainly, that the book does not cover it.

What it does

Drop in a PDF textbook and Prism Library breaks it into a searchable body of evidence: every chunk carries its heading hierarchy and its page number. You can then ask questions in plain language, and the AI answers strictly from the book — a direct answer, followed by numbered citations that open the exact source passage. If the answer is not in the book, it tells you "this textbook does not cover it" instead of improvising. The library, the index, and the retrieval all run locally, so your study material never leaves your machine.

How I built it

The pipeline has three stages, all in Python behind a pywebview desktop shell:

Parsing. PDFs are converted to structured Markdown (headings + page markers) by a local MinerU engine, with cloud OCR (Mistral / Qwen / GLM) as fallback for scanned books.
Indexing. Chinese text is pre-tokenized with jieba, then indexed in SQLite FTS5 with BM25 column weighting, so a keyword that appears in a heading outranks one buried in body text:

$$\text{BM25}(D,Q) = \sum_{i} \text{IDF}(q_i)\cdot\frac{f(q_i,D)(k_1{+}1)}{f(q_i,D)+k_1\left(1-b+b\frac{|D|}{\mathrm{avgdl}}\right)}$$

Semantic retrieval uses embeddings with cosine similarity, fused with the lexical results via reciprocal rank fusion:

$$\cos(\mathbf{u},\mathbf{v})=\frac{\mathbf{u}\cdot\mathbf{v}}{\lVert\mathbf{u}\rVert\,\lVert\mathbf{v}\rVert},\qquad \text{RRF}(d)=\sum_{r\in R}\frac{1}{k+\mathrm{rank}_r(d)}$$

Answering. Retrieved chunks are re-ranked by the model, then evidence extraction and "not covered" detection run as hard code, not prompt instructions — a model cannot be talked into skipping a citation, because the pipeline enforces it.

Physical page numbers and printed textbook page numbers are two different coordinate systems; I keep both and map between them with a two-point linear calibration:

$$y = y_1 + \frac{y_2-y_1}{x_2-x_1}(x-x_1)$$

Challenges I faced

Chinese full-text search is unforgiving. SQLite's trigram tokenizer refuses substrings shorter than three characters, which silently breaks most Chinese medical terms ("球菌", "形态"). I moved to jieba pre-tokenization, indexed section headings alongside body text, and separated chapter-level from section-level headings so the same term ("形态") appearing across many pathogens stops polluting every sub-section.
Three OCR engines, three heading trees. MinerU, Mistral, and GLM each flatten or re-level headings differently, so I wrote a normalization pass that rebuilds the heading hierarchy from numbering conventions — without it, "section 2 of Yersinia" was indistinguishable from "section 2" of any other genus.
Hallucination is the whole point. The hardest part was enforcing the no-hallucination rule. Evidence extraction, citation validation, and not-covered detection are all compiled into the pipeline, so the system prompt can change the tone of an answer but never the rule that every claim needs a source.

What I learned

The biggest lesson was that lexical retrieval has a ceiling, and trying to push past it with more threshold tuning only moved noise around. What actually fixed quality was an evaluation set of real textbook questions — it is the single source of truth I now run after every change to ranking or indexing. And building an AI tool for medicine taught me the discipline that matters most: when you cannot verify an answer, the honest move is to say so.

What's next

More textbooks and disciplines, a deployable web version so classmates on phones can use it, and a bring-your-own-key mode so every answer stays private and affordable at scale.