Document Q&A pipelines with PyPDF

The store operations handbook was 200 pages long. It covered fridge temperatures, opening checklists, refund rules, allergen labels and what to do when the oven alarm goes off. Every new hire was told to read it. Nobody did.

Instead, they phoned Meera. “Ma’am, the walk-in fridge says 6 degrees, is that okay?” “Ma’am, a customer wants ₹2,500 back for a squashed cake, can I approve it?” Three or four calls a week, often during the evening rush.

“I want a box on the staff tablet,” Meera told Arjun. “They type a question, it answers from the handbook, and it tells them which page. And if the answer isn’t in the handbook, it must say so. I do not want it making up fridge temperatures. That’s food safety.”

Arjun’s two shortcuts

Arjun had a Gemma model running from two lessons ago. His first idea: extract the whole handbook’s text, paste it into the prompt, and ask the question. He counted first, which was wise. The handbook was about 61,000 words. At roughly 1.3 tokens per English word, that is around 80,000 tokens. Gemma 3’s 1B model reads at most about 32,000 tokens at once: its context window, the amount of text a model can consider in one go, which you met in Module 01. The handbook was more than twice too big. Even a model with a large enough window would be slow and expensive, rereading 200 pages for every single question.

His second idea was to skip AI entirely: search the text for keywords. He typed refund. Forty-seven matches across nineteen pages. “Who approves a refund over two thousand?” needed a person to read all nineteen.

Priya sat down next to him. “When a new staff member asks you where something is in the handbook, what do you do?”

“I don’t reread the whole thing. I flip to the right section, skim a couple of pages, and read out the sentence.”

“Three steps, then. Find the right pages. Read them closely. Point at the answer. Let’s build exactly that.”

The plan, as a librarian would do it

Priya drew it on the whiteboard:

flowchart LR
  A["Handbook PDF"] --> B["PyPDF: text per page"]
  B --> C["Chunks that remember their page"]
  C --> D["Embeddings"]
  Q["Staff question"] --> E["Find closest chunks"]
  D --> E
  E --> F["Reader model points at the answer"]
  F --> G["Answer plus page, or I don't know"]

If you did Module 09, you will recognise this shape. It is RAG, retrieval-augmented generation: retrieve the relevant pieces first, then answer from them. The difference here is that every box runs on small open models from the Hugging Face Hub, on one laptop, with no API bills.

Think of it as a librarian with two skills. The first skill is knowing roughly where things are: “refunds are around page 12 and page 87”. That is retrieval. The second skill is close reading: given two pages, put a finger on the exact sentence. That is the reader. Neither skill alone is enough.

Following one question all the way through

Priya chose a question that new staff ask every week:

“Who approves a refund over 2,000 rupees?”

Step 1: text out of the PDF. PyPDF read all 200 pages and returned one string per page. Page 12 said, in part: “Store managers may approve refunds up to Rs 2,000 for damaged or late orders. Record every refund in the refund log before the end of the shift.” Page 87 said: “Refunds above Rs 2,000 must be approved by the area manager. The store manager should email the order number, photos and the customer’s complaint to the area manager, who replies within four hours.”

Step 2: chunks. Each page was cut into windows of about 120 words that overlap by 30, so a sentence cut at one boundary appears whole in the next window. 61,000 words with a step of 90 gives about 680 chunks. Every chunk keeps its page number.

Step 3: embeddings. An embedding is a list of numbers that captures what a piece of text means; texts with similar meanings get similar lists (Module 07). The small all-MiniLM-L6-v2 model turned each chunk into 384 numbers. That took about ten seconds for the whole handbook, once.

Step 4: retrieval. The question was embedded the same way, then compared with every chunk using cosine similarity (a score of how closely two lists of numbers point the same way, where 1 is identical and 0 is unrelated). The top four:

Rank	Page	Similarity	Chunk begins
1	12	0.668	“Store managers may approve refunds up to Rs 2,000…”
2	87	0.651	“Refunds above Rs 2,000 must be approved by the area manager…”
3	140	0.082	“Delivery partners must hand over the box upright…”
4	42	0.026	“Keep the walk-in fridge between 2 C and 4 C…”

Show the answer

No. Both chunks are about the same topic: refunds, Rs 2,000, who approves them. Embeddings capture topic and meaning very well, but they blur fine logical details like “up to” versus “above”. That is exactly why we retrieve several chunks (here four) instead of one, and then let a reader model, which looks at every word, make the final call. Retrieval narrows 680 chunks to 4. The reader picks the answer.

Step 5: the reader. The reader is an extractive question-answering model. Extractive means it never writes new words. It points at a span (a run of consecutive words) inside the text and says “the answer is from here to here”, like a student highlighting a textbook. We used deepset/roberta-base-squad2. The question and one chunk go in together:

<s> Who approves a refund over 2,000 rupees? </s></s> Refunds above Rs 2,000 must be approved by the area manager. ...

That is 57 tokens for page 87’s chunk. For every token in the chunk, the model outputs two scores: how likely the answer starts here, and how likely it ends here. The best span is the start and end pair with the highest combined score. It also scores one special position, the <s> token at the very beginning, which the model was trained to use to mean “no answer in this text”.

Here is what came back for the top two chunks:

Page	Best span	Span score	“No answer” score	Margin
12	“Store managers”	10.75	5.81	4.94
87	“the area manager”	13.69	6.39	7.30

The margin is the span score minus the “no answer” score: how much more the model believes in its answer than in “it’s not here”. Page 87 wins clearly. The system replied:

the area manager (handbook page 87)

Total time per question on Arjun’s laptop: well under a second.

Show the answer

Those confidences are softmax probabilities (the last lesson) computed within each chunk separately. Each one only says “of all the spans in this chunk, how strongly do I prefer this one?” Page 12 has one obvious candidate span, so it gets a high share even though the evidence is weaker. Comparing them across chunks is comparing percentages of different cakes. The raw scores, and especially the margin over “no answer”, live on the same scale for every chunk, so they can be compared. Many tutorials that loop over chunks and keep the “highest confidence” have exactly this bug.

And for an unanswerable question, “What colour is the delivery van?”, the best span on the fridge page was “cream” with a margin of −16.19. Strongly negative: the model is telling you plainly that the answer is not there. That is how you get an honest “I don’t know”.

Opening the hood: why PDFs are messy

A PDF is not a text file. It is a set of printing instructions. Inside a page you find commands like “use font F1 at size 12, move to position (72, 712), draw the string Keep the walk-in”. The file records where ink goes, not which words form a sentence or which column a line belongs to.

PyPDF is a pure-Python library that reads those instructions and rebuilds the text by following the drawing order and positions. Usually that works. Sometimes it cannot:

• Headers and footers (“Crumb & Co. Confidential, page 42”) are drawn on every page, so they appear in every page’s text.
• Two-column layouts can come out with lines from both columns interleaved, because the drawing order jumps between them.
• Tables lose their grid. A row like Walk-in fridge | 2–4 °C | check 3× daily becomes a jumble of words and numbers.
• Scanned pages are photographs of paper. There are no text instructions at all, so extract_text() returns an empty string. Those need OCR (optical character recognition: software that reads letters from an image), such as Tesseract.

Building it, one box at a time

Install everything first:

uv add pypdf sentence-transformers transformers torch

Box 1: extract, keeping page numbers.

from pypdf import PdfReader

reader = PdfReader("crumb_handbook.pdf")
pages = [(i + 1, page.extract_text() or "") for i, page in enumerate(reader.pages)]

empty = [n for n, text in pages if len(text.strip()) < 20]
print(f"{len(pages)} pages, {len(empty)} look empty (scanned?): {empty[:10]}")

reader.pages is the list of pages. extract_text() returns the page’s text, or an empty string. We store (page_number, text) pairs, counting from 1 the way humans do, and immediately report suspicious empty pages instead of discovering them later.

Box 2: strip repeated boilerplate, then chunk.

from collections import Counter

# Lines that appear on more than half the pages are headers or footers
line_counts = Counter(line.strip() for _, text in pages for line in set(text.splitlines()))
boilerplate = {line for line, n in line_counts.items() if line and n > len(pages) / 2}

def clean(text: str) -> str:
    return "\n".join(l for l in text.splitlines() if l.strip() not in boilerplate)

def chunk_words(text: str, size: int = 120, overlap: int = 30):
    """Yield windows of about `size` words that overlap by `overlap` words."""
    words = text.split()
    step = size - overlap
    for start in range(0, max(len(words) - overlap, 1), step):
        yield " ".join(words[start:start + size])

chunks = [
    {"page": page_no, "text": piece}
    for page_no, text in pages
    for piece in chunk_words(clean(text))
    if piece.strip()
]
print(len(chunks), "chunks")

The boilerplate trick is crude but effective: a line that appears on most pages is almost never the answer to anything. Footers with changing page numbers (“Page 42 of 200”) slip through; a small regular expression can catch those.

Box 3: embed once, search per question.

from sentence_transformers import SentenceTransformer, util

embedder = SentenceTransformer("all-MiniLM-L6-v2")   # small, fast, English
chunk_vecs = embedder.encode([c["text"] for c in chunks], convert_to_tensor=True)

def search(question: str, k: int = 4):
    q_vec = embedder.encode(question, convert_to_tensor=True)
    hits = util.semantic_search(q_vec, chunk_vecs, top_k=k)[0]
    return [chunks[h["corpus_id"]] | {"similarity": h["score"]} for h in hits]

for hit in search("Who approves a refund over 2,000 rupees?"):
    print(hit["page"], round(hit["similarity"], 3), hit["text"][:60])

util.semantic_search compares the question’s vector with every chunk’s vector and returns the top_k best, each with a corpus_id (the chunk’s position in our list) and a similarity score.

Box 4: the reader. Older versions of transformers offered pipeline("question-answering"), and many tutorials still use it. The current major version has dropped it, and in any case we want the raw scores. So we use the AutoModelForQuestionAnswering class from the last lesson directly. Its head outputs the start and end scores.

import torch
from transformers import AutoTokenizer, AutoModelForQuestionAnswering

QA_CKPT = "deepset/roberta-base-squad2"
qa_tok = AutoTokenizer.from_pretrained(QA_CKPT)
qa_model = AutoModelForQuestionAnswering.from_pretrained(QA_CKPT)

def read(question: str, context: str, max_answer_tokens: int = 30):
    """Find the best answer span in one chunk. Returns (answer, margin over 'no answer')."""
    inputs = qa_tok(question, context, return_tensors="pt",
                    truncation="only_second", max_length=384)
    with torch.no_grad():
        out = qa_model(**inputs)
    start = out.start_logits[0].tolist()   # one "answer starts here" score per token
    end = out.end_logits[0].tolist()       # one "answer ends here" score per token

    # Only spans inside the chunk count, not inside the question
    ctx = [i for i, s in enumerate(inputs.sequence_ids(0)) if s == 1]
    best = (float("-inf"), 0, 0)
    for i in ctx:
        for j in ctx:
            if i <= j < i + max_answer_tokens and start[i] + end[j] > best[0]:
                best = (start[i] + end[j], i, j)

    null_score = start[0] + end[0]         # position 0 means "no answer here"
    answer = qa_tok.decode(inputs["input_ids"][0][best[1]:best[2] + 1]).strip()
    return answer, best[0] - null_score

New pieces, line by line. truncation="only_second" trims the chunk, never the question, if the pair is longer than 384 tokens. sequence_ids(0) labels each token as belonging to the question (0), the chunk (1) or neither (special tokens), so we only consider spans inside the chunk. The double loop tries every start and end pair up to 30 tokens long and keeps the best. The margin compares that best span with position 0, the model’s “no answer” slot.

Box 5: put it together, with an honest “I don’t know”.

def ask(question: str, k: int = 4, min_margin: float = 2.0) -> str:
    best = None
    for chunk in search(question, k):
        answer, margin = read(question, chunk["text"])
        if best is None or margin > best["margin"]:
            best = {"answer": answer, "margin": margin, "page": chunk["page"]}
    if best is None or best["margin"] < min_margin:
        return "I couldn't find that in the handbook. Please ask your store manager."
    return f'{best["answer"]} (handbook page {best["page"]})'

print(ask("Who approves a refund over 2,000 rupees?"))
print(ask("What temperature should the walk-in fridge be?"))
print(ask("What colour is the delivery van?"))

the area manager (handbook page 87)
between 2 C and 4 C (handbook page 42)
I couldn't find that in the handbook. Please ask your store manager.

The min_margin of 2.0 is a starting point. Tune it on twenty real staff questions, including some whose answers are not in the handbook.

Library APIs change quickly. If a name has moved, check the docs linked below.

Quoting versus writing

Extractive answers are short fragments: “between 2 C and 4 C”. For questions like “How should I handle an angry customer?”, staff need a few full sentences that combine advice from several places. That is a job for generative answering: hand the retrieved chunks to an instruction-tuned model like the Gemma model from two lessons ago, and ask it to write an answer from them.

from transformers import pipeline

chat = pipeline("text-generation", model="google/gemma-3-1b-it", device_map="auto")

def ask_generative(question: str) -> str:
    context = "\n\n".join(f"[page {c['page']}] {c['text']}" for c in search(question))
    messages = [
        {"role": "system", "content": (
            "Answer only from the handbook excerpts. Cite pages like (page 42). "
            "If the excerpts do not contain the answer, say you don't know.")},
        {"role": "user", "content": f"Excerpts:\n{context}\n\nQuestion: {question}"},
    ]
    return chat(messages, max_new_tokens=150)[0]["generated_text"][-1]["content"]

	Extractive reader	Generative (Gemma)
Can invent facts	No: always a quote from the text	Sometimes
Honest “I don’t know”	Built in, via the margin	Only if the model obeys the prompt
Answer style	Short fragment	Full sentences
Combines several chunks	No	Yes
Speed on a laptop	Fast	Slower

Priya’s rule for Crumb: numbers, limits and safety facts come from the extractive reader. Advice and “how do I…” questions go to the generative path, with the pages shown so staff can check.

LangChain wraps the extraction step as PyPDFLoader (from langchain_community.document_loaders), which returns one document per page with the page number in its metadata. Underneath, it is PyPDF. For 200 pages, a Python list in memory is plenty; for 200,000 pages you would store the vectors in a vector database, as in Module 08.

Where it breaks

The tablet went live in the Koregaon Park store. For three weeks it was perfect. Then a store manager approved a ₹2,800 refund on her own, and when the area manager asked why, she showed him the tablet: “Store managers may approve refunds up to Rs 3,000.”

Head office had updated the handbook the week before, raising the limit. Arjun had replaced the PDF on the server. But the chunks and embeddings had been built at startup from the old PDF, and the server had not restarted since. Half the answers came from the new file, and the search index still pointed at the old one.

The fix was two lines and a habit: store a fingerprint of the PDF with the index, and rebuild whenever it changes.

import hashlib

def fingerprint(path: str) -> str:
    return hashlib.sha256(open(path, "rb").read()).hexdigest()
# Save fingerprint(...) next to the embeddings; on startup, rebuild if it differs.

Other failures experienced engineers check for:

• Answers split across chunks. If a rule begins on one page and finishes on the next, the overlap may not cover it. Overlap more, or chunk by section heading instead of word count.
• Tables. “What temperature for the display fridge?” fails when the answer lived in a table that extraction turned to soup. Extract tables separately, or rewrite key tables as sentences.
• Scanned appendices. The supplier contacts appendix was scanned, so every question about suppliers returned “I don’t know”. Run OCR on empty pages.
• Questions phrased unlike the text. “Can I give money back?” should match “refunds”. Embeddings usually handle this; test a few slang phrasings anyway.
• Threshold drift. Too low and the system answers questions it should not; too high and it refuses good ones. Re-check the threshold whenever you change models or chunk sizes.

Try it yourself

Take any PDF you have the right to use, such as a product manual, a rental agreement or a course syllabus. Build the extractive version and ask it six questions: three whose answers you know are in the document, and three whose answers are definitely not. Print each question’s margin.

Expected result: the three answerable questions come back with the right quote, the right page, and positive margins. The three unanswerable ones should come back with low or negative margins. Pick a min_margin that refuses all three bad questions while keeping all three good ones. If no single threshold does both, look at the retrieved chunks for the failures.

Stuck? Hint

Debug the boxes in order. Print search(question) first. If the right page is not in the top four, the problem is retrieval: try smaller chunks, a larger k, or rephrase. If the right page is there but the answer is wrong, print the chunk’s text: the PDF extraction may have scrambled it. Only then suspect the reader model.

Common mistakes

• Not looking at the extracted text. Garbled tables and empty scanned pages ruin everything downstream. Print a few pages before building anything else.
• Throwing away page numbers. Without them, nobody can check the answer. Attach the page to every chunk from the start.
• Comparing per-chunk probabilities. Compare raw span scores or margins across chunks, not softmax confidences.
• No “I don’t know” path. A confident wrong fridge temperature is a food safety problem. Always threshold, and test with unanswerable questions.
• Forgetting to re-index. When the document changes, rebuild the chunks and embeddings.

Recap

• PyPDF’s PdfReader(...).pages[i].extract_text() gets text out page by page; scanned pages need OCR.
• Chunk with overlap, strip repeated headers, and keep the page number on every chunk.
• Embed chunks once; for each question, retrieve the closest few. Retrieval narrows; the reader decides.
• An extractive reader scores every possible start and end, and a special “no answer” slot; the margin between them is an honest confidence you can threshold.
• Quote for facts and limits; generate for advice; always show the page.

On Friday Arjun demonstrated it to Meera on the staff tablet. “It works like a good librarian,” he said. “First it finds the four pages most likely to have the answer. Then a second model reads those pages closely and points at the exact words, with the page number. If it can’t find a clear answer, it says so, instead of guessing. And whenever head office changes the handbook, it rebuilds itself.”

Meera typed: “Who approves a refund over 2,000 rupees?” The tablet answered: “the area manager (handbook page 87)”. She looked satisfied. Then her phone buzzed with a two-minute WhatsApp voice note from the Baner store, half in Marathi, followed by a customer’s photo of a cake they wanted copied for a birthday.

“Text is only part of my bakery,” she said. “Can these models see? Can they hear?”

Go deeper

• Hugging Face Transformers documentation: https://huggingface.co/docs
• Hugging Face LLM course (question answering): https://huggingface.co/learn/llm-course
• LangChain (document loaders): https://python.langchain.com
• The original RAG paper: https://arxiv.org/abs/2005.11401