Every “chat with your PDF” tutorial works on the tutorial author’s PDF. Yours will include a scanned contract with no text layer, a two-column report whose lines interleave when extracted, a spreadsheet exported to PDF where the numbers arrive without their headers, and a four-hundred-page manual where the answer is on page 312. Those are four different problems and only one of them is about the model.
A PDF is not a document format
A PDF describes where to paint glyphs on a page. There is no paragraph, no reading order, no table and often no space character — extractors reconstruct all of that from coordinates. Everything difficult below follows from that one fact.
The practical consequence: extraction is a guess, and your pipeline should treat it as one. Log what it produced, keep the page number, and make it possible for a human to look at the original page when an answer looks wrong.
Sort your PDFs before you parse them
One cheap test decides the route: how much extractable text is on the page. Poppler’s pdftotext is on every Linux distribution and macOS via Homebrew, and it is the reliable way to ask.
- Run
pdftotext -layout -f N -l N file.pdf -for a page and count the non-whitespace characters. - Fewer than about 100 characters on a full page: treat it as scanned and send it to OCR.
- More than 100 but with a high ratio of single characters to words: suspect a broken encoding, and OCR it anyway.
- Otherwise use the extracted text — but keep
-layout, because without it a two-column page interleaves its columns line by line and produces sentences that read like a ransom note.
# classify.py
import re, subprocess
def page_text(path, page):
out = subprocess.run(
["pdftotext", "-layout", "-f", str(page), "-l", str(page), path, "-"],
capture_output=True, text=True, check=True)
return out.stdout
def page_count(path):
out = subprocess.run(["pdfinfo", path], capture_output=True, text=True, check=True)
return int(re.search(r"^Pages:\s+(\d+)", out.stdout, re.M).group(1))
def classify(path, page):
t = page_text(path, page)
letters = sum(c.isalnum() for c in t)
words = [w for w in t.split() if len(w) > 1]
if letters < 100:
return "scanned"
if words and letters / max(1, len(words)) < 2.0:
return "broken-encoding"
return "text"
Enter fullscreen mode Exit fullscreen mode
Classify per page, not per document. Mixed documents are the normal case in any organisation that has ever scanned a signature page and stapled it to a Word export.
Extraction, per class
Class Description text pdftotext -layout. Fast, free, deterministic. Keep the page number with every chunk. scanned Rasterise and OCR. Tesseract handles clean typed pages well and handwriting badly. broken-encoding The font has no usable ToUnicode map. OCR is the only route; the text layer is a lie. figure-heavy Charts and diagrams carry meaning no extractor gets. Send the page image to a vision model and store its description as text.# OCR route: rasterise at 300 dpi, then Tesseract.
pdftoppm -r 300 -f 12 -l 12 -png contract.pdf /tmp/page
tesseract /tmp/page-12.png /tmp/page-12 --psm 1 -l eng
cat /tmp/page-12.txt
Enter fullscreen mode Exit fullscreen mode
300 dpi is the floor for reliable OCR of 10-point text; 150 dpi roughly doubles the error rate on small print and 600 dpi buys little for four times the pixels. --psm 1 asks Tesseract to detect page layout and orientation, which is what you want for a scan that may be upside-down.
Tesseract’s options and language packs change between major versions, and vision-model APIs change faster than that. Check tesseract --help-extra and your provider’s current image input reference rather than trusting flags copied from a blog post — including this one.
For pages a text extractor mangles, a vision model reading the rendered page image often beats every parser, and the trade between OCR and a vision model comes down to cost per page against how structured the page is.
Tables, which break everything
A table extracted as text loses the association between a number and its column header, which is exactly the association a question is about. Three mitigations, in increasing order of cost:
- Keep the layout.
-layoutpreserves column spacing, so a model reading the chunk can often still see the grid. Free, and it works more often than you would expect. - Never split a table across chunks. Detect runs of lines with three or more multi-space gaps and treat the whole run as one atomic chunk, even if it exceeds your character budget. A half table is worse than a large chunk.
- Re-render the table as Markdown. Send the table region to a model with the instruction to reproduce it as a Markdown table and change nothing else, and store that alongside the raw text. Costs a call per table; makes numeric questions answerable.
Whichever you choose, repeat the header row into every chunk of a long table. A chunk of forty rows with no header is unanswerable and looks fine in a log.
The 400-page problem
Four hundred pages is roughly 200,000 tokens of prose. Even where a model’s context window swallows that, three things argue against it: you pay for all of it on every question, latency scales with it, and accuracy on facts placed in the middle of a very long context degrades measurably in the published literature.
Retrieval instead, with one addition that matters for long structured documents: keep the heading path. A chunk from page 312 that carries Chapter 9 > Maintenance > Hydraulic system in its metadata retrieves better and cites better than the same 1,200 characters alone.
# Track the heading stack while walking pages in order.
HEADING = re.compile(r"^\s{0,8}(\d+(?:\.\d+)*)\s+(\S.{0,80})$")
def chunks_with_headings(pages): # pages: list of (page_no, text)
stack, out = [], []
for page_no, text in pages:
for block in split_blocks(text):
m = HEADING.match(block.strip().splitlines()[0] if block.strip() else "")
if m:
depth = m.group(1).count(".")
stack = stack[:depth] + [m.group(2).strip()]
out.append({
"page": page_no,
"path": " > ".join(stack),
"text": block,
})
return out
Enter fullscreen mode Exit fullscreen mode
Then embed path + "\n" + text rather than the text alone. The heading is a handful of tokens and it puts the chunk in the part of the vector space its subject actually belongs to.
Answering with page numbers
A document answer without a page number is unverifiable, and an unverifiable answer about a contract is worse than no answer. Carry page through retrieval and require it in the output:
SYSTEM = (
"Answer only from the passages. After each claim, cite the page as (p. N) "
"using the page number given with the passage. If the passages do not "
"contain the answer, reply exactly: 'Not found in this document.'"
)
context = "\n\n".join(
"[p. " + str(h["page"]) + " | " + h["path"] + "]\n" + h["text"]
for h in hits
)
Enter fullscreen mode Exit fullscreen mode
Then verify cheaply: check that every (p. N) in the answer appears among the page numbers you actually supplied. A citation to a page that was not in the prompt is a fabricated citation, and it is detectable with a regex and a set difference rather than another model call.
What ingestion costs
400-page manual, ~500 words per page
= 200,000 words ~ 270,000 tokens
/ 300 tokens per chunk = ~900 chunks
Embedding at $0.02 per million tokens = 270,000 / 1e6 x $0.02 = $0.005
OCR with Tesseract (local) = $0, plus ~1-3 s per page of CPU
Vision-model fallback, 40 figure pages, if used
= 40 x (image tokens + prompt) — bill it at YOUR provider's image rate;
this is the term that dominates, and the one to measure before scaling.
Per question: 5 chunks x 300 tokens + question + answer ~ 2,000 tokens.
Enter fullscreen mode Exit fullscreen mode
The shape of that arithmetic is the point: ingesting text is nearly free, ingesting pictures is not, and questions are cheap. So the decision that controls your bill is how many pages get routed to the vision path — which is exactly what the classifier at the top of this page is for. How images are converted into tokens differs by provider, so put a real number in that line before you run a thousand documents through it.
Re-ingesting without redoing the expensive part
You will re-ingest. The chunker changes, the embedding model changes, the classifier turns out to be sending too many pages to the vision path. If ingestion is one function that goes from a PDF to vectors, every one of those changes costs a full OCR run over the whole corpus, and OCR is the slow part even when it is free.
So store the output of each stage separately, keyed on the input’s hash:
CREATE TABLE page_text (
doc_sha TEXT NOT NULL, -- sha256 of the PDF bytes
page INTEGER NOT NULL,
route TEXT NOT NULL, -- 'text' | 'ocr' | 'vision'
extractor TEXT NOT NULL, -- 'pdftotext-24.02' | 'tesseract-5.3' | model id
text TEXT NOT NULL,
at REAL NOT NULL,
PRIMARY KEY (doc_sha, page, extractor)
);
Enter fullscreen mode Exit fullscreen mode
The extractor column in the key is what makes this work. Upgrade Tesseract and the old rows stay, so you can compare the two extractions on the same page before committing — and roll back by changing which extractor the chunker reads, with no reprocessing at all.
- Changed the chunker? Re-chunk from
page_text. Free, seconds, no OCR. - Changed the embedding model? Re-embed from the chunks. Costs embedding tokens only, which is the cheap call.
- Changed the classifier? Re-run it over
page_textrow counts rather than the PDFs, and re-extract only the pages whose route changed. - New version of the document? A different
doc_sha, so nothing collides. Keep both and let retrieval filter by whichever version is current, or the answer to “what did the contract say in March” is lost.
Version-aware retrieval is worth designing in from the start even if you do not need it yet. Retrofitting it means re-ingesting everything, which is exactly the situation this section exists to avoid, and keeping an index in step with a changing corpus is harder than building it once.
답글 남기기