DocSage
Ask questions of your PDFs and get answers that cite the page they came from.
What it's for
A 90-page contract, a compliance report or a paper. Someone needs an answer they can check, not an answer that sounds right.
- Every claim in an answer carries its source and its page number.
- Says the documents do not cover it rather than filling the gap.
- Nothing leaves the machine: parsing, embedding, reranking and generation all run locally.
What it is
A local-first PDF question-answering system. It parses text, tables and scanned pages, retrieves with a hybrid of vector search and BM25, reranks with a cross-encoder, and streams an answer from a local model with inline citations and page numbers.
How it works
- Parsing PyMuPDF text, pdfplumber tables, Tesseract OCR
- Chunking 800 characters, 150 overlap, tables kept whole
- Embeddings all-MiniLM-L6-v2 into ChromaDB
- Hybrid retrieval Vector and BM25, fused with reciprocal rank fusion
- Reranking MiniLM cross-encoder keeps the best 5
- Answer Ollama llama3.2:3b, streamed over SSE
Engineering decisions
Built with
Case study
Hand a language model a 90-page PDF and it will answer anything you ask, immediately, in a confident voice, whether or not the document says so.
For a contract, a paper or a compliance report, “sounds right” is worthless. You need the sentence and you need to know which page it came from, so you can go and check.
What goes wrong with one retriever
Vector search finds paraphrases. Ask about “termination” and it surfaces the clause about ending the agreement, which keyword search would have missed.
It is also the reason people distrust these systems, because it cheerfully misses exact strings. Search for a part number, a standard like EN 388, or a surname, and the nearest-neighbour match is often a passage that’s merely about the same topic. BM25 nails those and misses the paraphrases.
So DocSage runs both and merges them with reciprocal rank fusion: a passage ranked well by either method gets promoted, and no calibration between two incomparable score scales is needed. It’s the same fusion idea as the retriever in the ICACIN 2026 paper I co-authored, which is where I’d argue about it at more length.
A cross-encoder then rescores the merged candidates and keeps the best five. Fusion decides what’s worth a second look; the reranker decides what actually goes to the model.
The pipeline
- Parse. PyMuPDF for text, pdfplumber to pull tables out intact, Tesseract OCR for pages that come back nearly empty, so scanned documents still answer.
- Chunk. 800 characters with 150 of overlap. Tables stay whole, because a table cut in half is worse than no table, and document metadata gets a chunk of its own.
- Embed. all-MiniLM-L6-v2 into ChromaDB, with document records in SQLite.
- Retrieve. Top 20 by vector, top 20 by BM25, fused, reranked to 5.
- Answer. llama3.2:3b in Ollama at temperature 0.1, streaming over SSE, citing each claim as [Source N] with its page.
Nothing leaves the machine: embeddings, reranking and generation all run locally.
Local is not a preference here, it is what makes the tool usable at all. The documents worth asking questions of are contracts, compliance reports and unpublished papers, and those are exactly the documents nobody is allowed to paste into a hosted model. A per-document API cost also turns “index everything and see” into a decision, which is the wrong shape for a research tool.
Why it refuses
The prompt tells the model to say the documents don’t cover it rather than fill the gap. A system that answers everything is indistinguishable from one that’s guessing, and the refusals are what make the answers worth anything.
What isn’t measured yet
There are no saved evaluation results. scripts/evaluate.py measures latency but nothing has been recorded, so there’s no honest number to put on this page yet.
What isn’t measured yet
There are no saved evaluation results. scripts/evaluate.py measures latency but nothing has
been recorded, so there is no honest number to put on this page for answer time or for the
size of the largest document indexed.
The number that matters most is also missing: of a set of questions the documents genuinely do not answer, how many does it refuse. The refusal behaviour is the claim this whole system rests on, and right now it is an argument from design rather than a measurement.
Two loose ends worth fixing while in there: rank_bm25 and pdfplumber are imported but
missing from requirements.txt, so a clean install will not run.
Two loose ends worth fixing while you’re in there: rank_bm25 and pdfplumber are imported but missing from requirements.txt, so a clean install won’t run.