privatae-bastion/client-go/contextocr.go

package main

// OCR — the literal words on a picture, read on the USER'S machine.
//
// It is the largest remaining gap by volume after PDF
// text, and it is the one where getting it wrong is worst: a scanned document
// is images wrapped in a PDF, so pdftotext returns nothing while the page is
// full of words. A user who uploads a scanned contract and gets back an empty
// result has been told, silently, that their document says nothing.
//
// NOTE the boundary the picture lane drew and this file stays on the right side of:
// deterministic OCR of literal on-screen text is NOT "describe the picture with
// a vision model and embed the caption". That remains a user-enabled option and
// must never be presented as this. Nothing here calls a model.
//
// ── THE PROBE RUNS A REAL OCR, and this is the measured reason ──────────────
//
// tesseract ships in two pieces on every distro that packages it: the engine
// and the LANGUAGE DATA. Ubuntu has `tesseract-ocr` and `tesseract-ocr-eng` as
// separate packages (verified on this box: dpkg -L tesseract-ocr lists no
// .traineddata at all). A box with the engine and no language data cannot read
// one word.
//
// MEASURED, with TESSDATA_PREFIX pointed at an empty directory:
//
//	tesseract --version       exit 0      <- says nothing about language data
//	tesseract --list-langs    exit 0      <- prints a list of ZERO and exits 0
//	tesseract <img> - --psm 7 exit 1      <- the only one that tells the truth
//
// So the house probe idiom — run the tool, take the exit code (detectRuntime,
// harness.go:580) — is NOT sufficient here, because the obvious argv to run
// exits 0 on a machine that cannot OCR. Both cheap probes are PROXIES for the
// capability; the capability itself is "can this engine turn a picture of a
// word into that word", and the only authority for it is doing exactly that.
//
// Hence ocrProbePNG: a 180x44 grey PNG of one known word, OCR'd for real on
// every probe. 0.05 s measured, cached for prepCacheFor like every other probe.
// The precedent is the platform's own media capability gate, which embeds a
// real 16x16 PNG rather than trusting a model's self-description
// (the host's media engine) — and for the same reason: a
// capability is what the thing DOES.
//
// `osd` is excluded from the language list deliberately. It is orientation and
// script detection, it ships inside the base `tesseract-ocr` package, and it
// cannot recognise text — so a box with only osd.traineddata reports "1
// language" and still OCRs nothing. Counting it would rebuild the exact bug
// this file exists to prevent.

import (
	_ "embed"
	"fmt"
	"os"
	"path/filepath"
	"sort"
	"strings"
	"time"
)

// ocrProbePNG is a 1,975-byte grey PNG reading one word, generated by
// rasterizing a one-line PostScript page at 72 dpi (the smallest resolution at
// which this build's tesseract still reads it cleanly — 72/100/150 dpi were all
// tried and all read it, so the smallest was taken). It is never stored, never
// sent anywhere, and exists only to be OCR'd.
//
//go:embed ocrprobe.png
var ocrProbePNG []byte

// ocrProbeWord is what the probe image says. Required back from the engine when
// the box is OCRing in English, which is the case on the overwhelming majority
// of machines and the only case where a specific answer is safe to demand: a
// Latin-script model reads this correctly whatever language it was trained for,
// but a model trained on a non-Latin script has no honest obligation to.
const ocrProbeWord = "PRIVATAE"

// ocrProbePSM is "treat the image as a single text line" — right for a
// one-word image, and NOT what real extraction uses (that gets the default
// full-page segmentation). The probe is proving the engine loads and
// recognises, not that page layout analysis works.
const ocrProbePSM = "7"

// ocrLangOSD is tesseract's orientation/script detector. It appears in
// --list-langs, it is not a recognition language, and it arrives with the
// engine rather than with a language pack. See the header.
const ocrLangOSD = "osd"

// ocrDPI is what a scanned PDF's pages are rasterized at before OCR.
//
// 300 is the number every production document pipeline assumes: it is
// tesseract's own documented floor for reliable recognition, it is what AWS
// Textract, Google Document AI and Azure Document Intelligence all state as the
// resolution they expect scanned input at, and 8.4 megapixels for a 300-dpi
// letter page is the figure OCRmyPDF uses when it reasons about page size.
//
// MEASURED on this box, on a real image-only PDF whose embedded scan is 200 dpi
// (ghostscript -sDEVICE=pdfimage24), one letter page:
//
//	150 dpi   1275x1650   0.36 s   same text, every line exact
//	200 dpi   1700x2200   0.44 s   same text, every line exact
//	300 dpi   2550x3300   2.18 s   same text, every line exact
//	400 dpi   3400x4400   2.39 s   same text, every line exact
//
// Accuracy was identical at all four — on CLEAN synthetic input, which is the
// limitation of that measurement and the reason it does not decide the number.
// Real scans carry sensor noise, skew and JPEG ringing, which is exactly the
// regime where resolution stops being free, and the published floor is 300.
// Six times the CPU for a document the user will read once is the cheap side of
// that trade; a wrong transcription of their contract is not.
const ocrDPI = "300"

// ocrPageCap is how many pages of one scanned PDF a single job OCRs.
//
// This lane is SYNCHRONOUS — one management dispatch, one answer — so the cap
// is what keeps a 400-page scan from holding a worker on the user's own machine
// for twenty minutes. Production draws the same line in the same place and
// draws it LOWER: Google Document AI accepts 15 pages for online processing
// (and 500 in batch), AWS Textract accepts ONE page synchronously for PDF (3000
// asynchronously), Azure Document Intelligence 2 pages on its free tier. 30 is
// twice the most generous of those, and it covers the documents a person
// actually scans: an invoice, a receipt, a letter, an ID, a contract.
//
// It is REPORTED, never silent: the reply says how many pages were read and
// that there were more, so a user is never left to infer that page 31 was
// blank. Raising it is a change to ocrExtractTimeout as well — the two are one
// decision, and ocrPerPageBudget is the arithmetic between them.
const ocrPageCap = 30

// ocrPerPageBudget is the time one page is allowed to cost, and it is where
// ocrExtractTimeout comes from rather than being a second guess at the same
// number. MEASURED above at 2.18 s wall for a 300-dpi letter page plus 0.55 s
// to rasterize it; 4 s is that with room for a machine slower than this one,
// which a bastion frequently is.
const ocrPerPageBudget = 4 * time.Second

// ocrSetupBudget is the fixed cost before per-page work begins: process start,
// the PDF's own parse, and the text-layer pass that decides which pages need
// OCR at all.
const ocrSetupBudget = 30 * time.Second

// ocrExtractTimeout bounds a whole pdf-ocr job. Derived, not chosen:
// 30 + 30x4 = 150 s.
const ocrExtractTimeout = ocrSetupBudget + ocrPageCap*ocrPerPageBudget

// ocrImageTimeout bounds a single-image OCR. One picture, no rasterizing, and
// measured at 0.64 s for a full 300-dpi page — 30 s is the allowance for an
// unusually large scan on an unusually slow machine, not an expected cost.
const ocrImageTimeout = 30 * time.Second

// ocrPageHasText is how few characters a page's TEXT LAYER may hold before
// pdf-ocr gives that page to the engine as well and compares the two readings.
//
// IT IS A BUDGET, NOT A CLASSIFIER, and that is the correction. It used to be
// 16, chosen on the claim that there is "no ambiguous middle" between a scanned
// page and a text page. There is, it is common, and it is a STAMP: a copier
// footer, a fax TTI header, a Bates number or an e-signature block writes a few
// real characters into the text layer of a page that is otherwise a picture.
// Under 16 such a page read as "this page has text" and its body never reached
// the engine, so the user got the name of the app that scanned their contract
// and none of their contract.
//
// THE TWO POPULATIONS OVERLAP, so no character count separates them. MEASURED
// on this box:
//
//	SPARSE BUT GENUINE TEXT PAGES — 203 pages of 9 real documents (a published
//	specification and eight papers), chars per page through pdftotext -layout:
//	    minimum 48 (a title page), then 110, 116, 268, 344, 364; median 2,340.
//
//	A STAMP OVER A SCAN — 11 real stamp/header/footer formats written over an
//	image-only page: 12 (CONFIDENTIAL), 18 (a Bates number), 19, 21 (a scanner
//	app footer — the reported case), 35, 36, 57 (an ITU-T T.30 fax TTI header),
//	58 (a DocuSign envelope footer), 61, 62, 99 (a three-line e-sign audit
//	block). Minimum 12, median 36, maximum 99.
//
// 48 < 99, so the populations overlap and every threshold misclassifies one side
// or the other. Apache Tika, the closest production precedent for exactly this
// decision, sets its equivalent to 10 (OcrConfig.java, StrategyAuto.BETTER =
// new StrategyAuto(10, 10); "if the total characters on the page <
// totalCharsPerPage ... then OCR will be performed") — below ours, and so it
// does not separate them either.
//
// WHAT DOES SEPARATE THEM is not the count, it is the ratio between the two
// readings of the SAME page, which is why this number no longer has to be a
// classifier: see ocrBeatsLayer. This one only decides who gets a second
// opinion, and the errors either side of it are:
//
//	too HIGH -> a sparse but genuine text page is rendered and OCR'd for
//	            nothing. Cost: CPU, measured below, and no wrong text, because
//	            ocrBeatsLayer decides what is kept.
//	too LOW  -> a stamped scan is declared read. Cost: the user is told their
//	            document says "Scanned by CamScanner". That is the single worst
//	            outcome in this lane and the reason the lane exists.
//
// THE NUMBER, derived rather than picked. It must clear the whole measured stamp
// band (maximum 99) with room, and stop below the point where margin text stops
// being a margin artifact. A stamp is at most three lines — the most verbose
// real format measured, an e-signature audit block, is three — and a full line
// across a letter page's 6.5-inch printable width holds 104 characters at 8 pt,
// the smallest size a stamp is realistically set in (measured with ghostscript
// `stringwidth` against real Helvetica metrics: 104.2 at 8 pt, 92.6 at 9, 83.4
// at 10, 69.5 at 12). Three full lines at 8 pt is 312.
//
// THE CPU IT SPENDS, measured on the same 203 real pages, at 0.49 s to render a
// page at 300 dpi plus 2.18 s to OCR it:
//
//	band     pages in band       extra on a 20-page text document
//	  16      0 / 203 (0.0%)        0.00 s
//	 128      3 / 203 (1.5%)        0.79 s
//	 312      4 / 203 (2.0%)        1.05 s
//	 512      7 / 203 (3.4%)        1.84 s
//
// against the 9.8 s of unconditional rendering the same document paid before the
// layer-first restructure below. So the widening costs about a ninth of what the
// restructure gives back, and a text document is still far cheaper than it was.
const ocrPageHasText = 312

// ocrBeatsLayer is how many times longer the engine's reading of a page must be
// before it is kept in place of the page's own text layer.
//
// A MULTIPLE RATHER THAN A CHARACTER, because widening ocrPageHasText is what
// makes the multiple necessary. The band now admits genuinely sparse text pages,
// and on those the layer and its OCR come out close in length, so "keep
// whichever is longer" is a coin flip decided by noise. MEASURED on
// six real sparse pages from real documents (48 to 364 characters of layer), the
// OCR-to-layer length ratio was 0.92, 1.08, 0.10, 0.81, 0.84 and 0.42 — and at
// 1.08 the length rule DID flip one: a 110-character layer lost to a
// 119-character OCR that kept 11 of the layer's 16 words and invented 8 the page
// does not contain.
//
// On the other side, the same measurement over five real stamp formats written
// on a full-page scan (2,530 characters of body) gave 26.7, 44.9, 45.7, 121.6
// and 141.7. Nothing measured sits between 1.08 and 26.7, so 2 is inside a gap
// 25 times wide: it is 1.85x above the highest genuine-text ratio and 13x below
// the lowest stamped-scan ratio.
//
// WHAT IT STILL GETS WRONG, stated rather than glossed: a scan so sparse that
// its body OCRs to less than twice its stamp — a receipt with a long footer —
// keeps the stamp. Length is a proxy either way; this narrows the window in
// which the proxy is uninformative rather than closing it.
const ocrBeatsLayer = 2

// ocrLangs is every recognition language this machine's engine can actually
// load, sorted, with osd excluded. Empty means the engine is installed and has
// no language data — a capability answer, not an error.
func ocrLangs() []string {
	out, err := runCmd(5*time.Second, "tesseract", "--list-langs")
	if err != nil {
		return nil
	}
	// The first line is a header ("List of available languages in ..."); the
	// rest are the language codes, one per line, on STDOUT (measured — unlike
	// pdftotext's version banner, which is on stderr).
	langs := []string{}
	for i, line := range strings.Split(out, "\n") {
		line = strings.TrimSpace(line)
		if i == 0 && strings.Contains(line, "available languages") {
			continue
		}
		if line == "" || line == ocrLangOSD || strings.Contains(line, " ") {
			continue
		}
		langs = append(langs, line)
	}
	sort.Strings(langs)
	return langs
}

// ocrLang is the language this box OCRs in: English when it has it, otherwise
// the first it does have.
//
// One language, not all of them joined with '+'. tesseract accepts `-l eng+deu`
// and is slower and less accurate for it, and a box with five packs installed
// would pay that on every page for no one's benefit. Which one it chose travels
// to the host in the tool report, so nothing about the choice is hidden — and
// the box's whole language list travels too, so a later slice can let the host
// name one.
func ocrLang() string {
	langs := ocrLangs()
	for _, l := range langs {
		if l == "eng" {
			return "eng"
		}
	}
	if len(langs) > 0 {
		return langs[0]
	}
	return ""
}

// ocrProve is the capability decision for every OCR job: read the known word
// off the embedded image, for real, with this box's own language.
//
// Returns (false, reason) rather than (false, "") on purpose — "tesseract is
// here and has no language data" is a DIFFERENT answer from "there is no
// tesseract", the user fixes them with different commands, and the scan report
// carries the distinction to the host. That is the quiet-vs-blind rule this
// heartbeat already applies three times (agents.go:423-444) applied to a tool
// instead of to a scan.
func ocrProve() (bool, string) {
	lang := ocrLang()
	if lang == "" {
		return false, "tesseract is installed but has no language data — it cannot " +
			"read a single word until a language pack is installed (on Debian/Ubuntu " +
			"that is a separate package, e.g. tesseract-ocr-eng; elsewhere it is a " +
			".traineddata file in the tessdata directory)"
	}
	dir, err := os.MkdirTemp("", "ocrprobe-")
	if err != nil {
		return false, "tesseract is installed but there was no scratch space to prove it works: " + err.Error()
	}
	defer os.RemoveAll(dir)
	img := filepath.Join(dir, "probe.png")
	if err := os.WriteFile(img, ocrProbePNG, 0o600); err != nil {
		return false, "tesseract is installed but the probe image could not be written: " + err.Error()
	}
	out, err := runCmd(15*time.Second, "tesseract", img, "-", "-l", lang, "--psm", ocrProbePSM)
	if err != nil {
		return false, fmt.Sprintf("tesseract is installed with language %q but failed to "+
			"read a one-word test image — the engine cannot OCR on this machine", lang)
	}
	got := strings.TrimSpace(out)
	if got == "" {
		return false, fmt.Sprintf("tesseract ran with language %q and returned nothing for a "+
			"one-word test image — the engine is not recognising text on this machine", lang)
	}
	// The strict check, applied only where it is safe. A Latin-script engine
	// reads this word whatever language it was trained for; a non-Latin one has
	// no obligation to, so demanding it there would refuse a box that works.
	if lang == "eng" && !strings.Contains(strings.ToUpper(got), ocrProbeWord) {
		return false, fmt.Sprintf("tesseract read %q from a test image that says %q — "+
			"the English language data on this machine is not working", firstLine(got), ocrProbeWord)
	}
	return true, ""
}

// ocrPSMAuto is "automatic page segmentation WITH orientation and script
// detection" — tesseract's psm 1, and the one flag that makes an upside-down
// scan readable.
//
// WHY IT IS HERE. A duplex feeder routinely puts pages through the other way up,
// and until this flag a 180-degree page went into the user's corpus as garbage
// with nothing anywhere saying so. MEASURED on this box (tesseract 5.5.0,
// leptonica 1.86.0), a 300-dpi raster of a six-line letter page:
//
//	rotation   default psm (3)                  --psm 1
//	0          exact                            EXACTLY THE SAME BYTES
//	90         exact (layout analysis fixes it) exact
//	180        "'y yBnoswuy) ind sem yeu]…"     exact
//	270        "MO|9q pe]uULd SSeippe…"         exact
//
// WHY THIS RATHER THAN A --psm 0 PRE-PASS, which is the other way to do it:
// psm 0 is a second exec per page that reports an orientation and leaves the
// caller to rotate the raster and pick a confidence floor. Measured, the floor
// is the problem — across seven real pages the confidence numbers do not
// separate the cases at all (genuinely upright pages scored 1.32, 2.18, 12.34
// and 40.10; genuinely rotated ones scored 1.58, 5.44, 6.24 and 12.40), so any
// threshold picked from this evidence would be a number with nothing behind it,
// and a misfire ROTATES A PAGE THAT READS FINE TODAY. psm 1 hands that judgment
// to tesseract's own internal threshold, which is the authority for it.
//
// WHAT IT DOES NOT FIX, stated rather than glossed: on a page with very little
// text — a receipt, a page with two lines on it — OSD answers "Too few
// characters. Skipping this page" and the page is read upright, exactly as it is
// today. So this is a strict improvement and never a regression, and a sparse
// upside-down page is still garbage. Fixing that one needs a confidence floor
// this measurement cannot supply.
//
// COST: +0.17 s per page measured (1.88 s to 2.72 s over five runs of the same
// page), against ocrPerPageBudget of 4 s.
//
// A BOX WITHOUT osd.traineddata STILL WORKS. Measured with TESSDATA_PREFIX
// pointed at a directory holding only eng: tesseract prints "Warning: Auto
// orientation and script detection requested, but osd language failed to load"
// on stderr, reads the page anyway, and EXITS 0. osd ships inside the base
// tesseract package on Debian/Ubuntu rather than with a language pack, so the
// common case is that it is there; the uncommon one degrades to today's
// behaviour instead of failing.
const ocrPSMAuto = "1"

// ocrArgv reads one image file. `-` as the output base is tesseract's spelling
// of stdout (verified against the real binary, not documentation).
func ocrArgv(in string) []string {
	lang := ocrLang()
	if lang == "" {
		// Cannot happen through the advertised path (the probe refuses first),
		// and if it somehow does, tesseract's own default is eng and its own
		// error message is better than a fabricated one.
		lang = "eng"
	}
	return []string{in, "-", "-l", lang, "--psm", ocrPSMAuto}
}

// ── pdf-ocr: the scanned half, and why it is ONE job and not a fallback ─────
//
// A scanned PDF is images wrapped in a PDF. pdftotext returns nothing for it
// while the page is covered in words, so SOMETHING has to notice and reach for
// the engine. There were three places to put that decision and only one of them
// survives contact with the rest of this lane:
//
//	THE BOX SILENTLY UPGRADES pdf-text INTO OCR. Refused. The host asked for
//	one job and would be billed a hundred times the CPU for another, on the
//	user's machine, with `extracted_by` recording something that did not
//	happen. A box that does work it was not asked for is the same dishonesty as
//	a box that advertises work it cannot do.
//
//	THE HOST RUNS pdf-text, SEES EMPTY, AND RE-DISPATCHES OCR. Refused, and for
//	a concrete reason rather than a stylistic one: the pull ticket is
//	SINGLE-USE and bound to one machine (the host's ticket store), so
//	a second attempt means a second ticket and a second download of the same
//	document — up to 64 MiB pulled twice over the user's own link to read it
//	once.
//
//	THE HOST ASKS FOR THE JOB IT WANTS, BY NAME. Taken. `pdf-ocr` is a distinct
//	advertised job whose contract is "this PDF's text, reading the pages that
//	have none" — text layer first, engine only where the text layer is empty.
//	The host asks for it when the machine advertises it, gets it in one
//	dispatch on one ticket, and the name it asked for is the name that ran.
//
// The honesty this buys is the point: a box advertises pdf-ocr only when it has
// BOTH poppler's rasterizer and an engine that provably reads a word, so the
// host never offers OCR on a machine that would refuse it, and a machine with
// poppler alone still offers pdf-text. The three answers a user can get stay
// three different sentences — no OCR here, an engine with no language data, and
// a document whose pages really are blank.
//
// ── THE ORDER: TEXT LAYER FIRST, and the reason the old order was wrong ─────
//
// This job used to render every page before it read a single character, and the
// justification written here was that the extra rendered page doubled as the
// page-count probe, so pdf-ocr needed two binaries instead of three. That
// justification did not survive being measured. MEASURED with poppler
// 26.01.0, one two-page text document, medians of ten and five runs:
//
//	pdftotext -layout -q -f 1 -l 31 -- doc.pdf -     6.2 ms   exit 0, exactly 2 form feeds
//	pdftoppm  -png -gray -r 300 -f 1 -l 3 doc.pdf  978.5 ms   2 PNGs
//
// pdftotext takes the SAME cap+1 bound, exits 0 when asked for more pages than
// the document has, and emits exactly one form feed per real page — including
// for an image-only PDF, which comes back as form feeds and nothing else. So the
// layer pass carries the page count for free, on a binary this job already runs,
// and no third binary is needed either way. The old order taxed every
// born-digital document roughly 160x for a count it could have had for 6 ms.
//
// The order now is: read the layer for cap+1 pages, take the count from it, and
// render ONLY the pages whose layer is under ocrPageHasText. A text document
// renders nothing at all. A scan does exactly what it did before plus one 6 ms
// layer pass, because none of its pages has a layer to keep.
//
// WHEN POPPLER WILL NOT ANSWER AT ALL, the count goes with the text — so there
// is a fallback to the old render probe, and only then. See renderPageProbe.

// ocrPageResult is one page of a scanned PDF and how its text was obtained.
type ocrPageResult struct {
	Page int    `json:"page"`
	How  string `json:"how"` // "text" (the PDF's own layer) or "ocr"
	Text string `json:"-"`
}

// runPDFOCR is the pdf-ocr job: the PDF's text layer where it has one, the OCR
// engine where it does not.
//
// Returns (text, timedOut, err) to match runExtract, plus the per-page record
// so the reply can say what was actually done rather than leaving the user to
// infer it.
func runPDFOCR(root, in string, deadline time.Time) (string, []ocrPageResult, int, bool, error) {
	return runPDFOCRCapped(root, in, deadline, ocrPageCap)
}

// runPDFOCRCapped is runPDFOCR with the page cap passed in, so a test can drive
// the more-pages path against a two-page document instead of building and OCRing
// a thirty-one page one. The cap is the only thing that varies; every other
// decision in here is the one production takes.
func runPDFOCRCapped(root, in string, deadline time.Time, pageCap int) (string, []ocrPageResult, int, bool, error) {
	dir, err := os.MkdirTemp(root, "ocrpages-")
	if err != nil {
		return "", nil, 0, false, fmt.Errorf("no scratch space for the pages: %w", err)
	}
	// The user's document, one page at a time, on their own disk. Gone before
	// the reply, whatever happened — including a panic, which unwinds through
	// this.
	defer os.RemoveAll(dir)

	// THE TEXT LAYER FIRST, for ONE PAGE MORE than the cap.
	//
	// The cap+1 trick is unchanged; only the pass it rides on has moved. The
	// layer pass answers "how many pages" from the number of form feeds it
	// emits, which it does for a scanned document too — an image-only PDF comes
	// back as one form feed per page and nothing else (measured). So the count
	// costs nothing extra and no page has to be rendered to obtain it.
	layer, layerRead := pdfTextByPage(in, pageCap+1, deadline)

	var (
		rendered []string // pre-rendered pages, only on the fallback path below
		total    int
		more     bool
	)
	if layerRead {
		total = len(layer)
		if total > pageCap {
			more = true
			total = pageCap
			layer = layer[:pageCap]
		}
	} else {
		// POPPLER WOULD NOT READ THE LAYER, so there is no page count either and
		// the job falls back to the render probe this used to do unconditionally.
		// Every page then goes to the engine, which is the safe direction and
		// exactly today's behaviour for a document with no readable layer.
		layer = nil
		rendered, more, err = renderPageProbe(dir, in, pageCap, deadline)
		if err != nil {
			return "", nil, 0, false, err
		}
		if rendered == nil {
			return "", nil, 0, true, nil // out of time before anything was rendered
		}
		total = len(rendered)
	}
	if total == 0 {
		return "", nil, 0, false, fmt.Errorf("poppler found no pages in this PDF")
	}

	results := make([]ocrPageResult, 0, total)
	// BOUNDED AS IT ACCUMULATES, for the reason runExtract's header records:
	// runExtract caps each PAGE at ctxExtractMaxText, but thirty of those added
	// together is thirty times the cap held in one process on the user's machine.
	// Past the ceiling the pages keep being READ and counted — the page record
	// stays honest — and their text stops being kept, which the reply then
	// reports as truncated exactly as it does for a single oversize document.
	b := &capWriter{max: ctxExtractMaxText + 1}
	for i := 0; i < total; i++ {
		if time.Now().After(deadline) {
			// OUT OF TIME IS NOT AN EMPTY DOCUMENT. Report what was read and
			// say the clock ran out; the caller turns that into a partial
			// answer rather than a refusal, because the pages already read are
			// the user's and throwing them away helps nobody.
			return b.buf.String(), results, total, true, nil
		}
		if len(strings.TrimSpace(layerAt(layer, i))) >= ocrPageHasText {
			// Above the band: this page's own text is the answer and it is
			// never rendered. On a born-digital document that is every page.
			results = append(results, ocrPageResult{Page: i + 1, How: "text"})
			b.Write([]byte(layer[i] + "\n"))
			continue
		}
		// Inside the band, so this page gets a second opinion — which means it
		// has to be rendered now, one page at a time. Rendering per page rather
		// than in one pass costs ~8 ms of process start per page (measured: 493
		// ms for one page, 978 ms for two, at 300 dpi) against the ~490 ms the
		// page itself costs, and it is what lets a document render only the
		// pages that need it rather than all of them.
		p := ""
		if i < len(rendered) {
			p = rendered[i]
		} else {
			pageBudget := time.Until(deadline)
			if pageBudget <= 0 {
				return b.buf.String(), results, total, true, nil
			}
			p, err = renderOnePage(dir, in, i+1, pageBudget)
			if err != nil {
				// This page could not be drawn. NOT the document's failure and
				// not a blank page: keep whatever layer it had, say the page was
				// not read when it had none, and carry on with the rest — which
				// are the user's and are fine.
				if l := strings.TrimSpace(layerAt(layer, i)); l != "" {
					results = append(results, ocrPageResult{Page: i + 1, How: "text"})
					b.Write([]byte(layer[i] + "\n"))
					continue
				}
				results = append(results, ocrPageResult{Page: i + 1, How: "unread"})
				continue
			}
		}
		// ONE PAGE CANNOT EAT THE WHOLE DOCUMENT'S BUDGET. Bounded at three
		// times the per-page allowance rather than at all the time remaining:
		// a single pathological page — a full-bleed photograph the engine
		// grinds over — would otherwise consume the 150 s and every page after
		// it would be reported as "ran out of time", which blames the document
		// for one page's cost. Whichever is smaller still applies, so the
		// overall deadline is never exceeded.
		pageBudget := 3 * ocrPerPageBudget
		if left := time.Until(deadline); left < pageBudget {
			pageBudget = left
		}
		out, pageTimedOut, oerr := runExtract(pageBudget, "tesseract", ocrArgv(p)...)
		text := strings.TrimSpace(out)
		if pageTimedOut && text == "" {
			// NOT READ, and said so — distinct from a blank page, which is the
			// whole distinction this lane is built around. The document keeps
			// going: the other pages are fine and they are the user's.
			results = append(results, ocrPageResult{Page: i + 1, How: "unread"})
			continue
		}
		// KEEP THE TEXT LAYER UNLESS THE ENGINE FOUND SEVERAL TIMES MORE. This is
		// the whole comparison, and it runs on every page rather than only on the
		// pages where OCR came back empty.
		//
		// It used to run only on the empty case, while the constant above and
		// this paragraph both promised the comparison. A page with a short real
		// text layer — a reference code, a stamp, a caption — and a page image
		// the engine reads badly would then have its REAL characters thrown away
		// for the engine's worse guess, and the record said pages_ocr=1 as though
		// a choice had been made. Reproduced: a 13-character layer against an
		// 8-character OCR kept the 8 and discarded the 13.
		//
		// LENGTH IS THE MEASURE, and it is a proxy rather than a truth — there is
		// no way on this box to ask which of two strings is the better reading of
		// a page. The MULTIPLE is what makes the proxy usable across the whole
		// band this now admits: see ocrBeatsLayer for the two measured
		// populations it sits between. A page whose layer is empty has nothing to
		// compare against, so any reading wins it.
		if oerr != nil || len(text) < ocrBeatsLayer*len(strings.TrimSpace(layerAt(layer, i))) {
			if l := strings.TrimSpace(layerAt(layer, i)); l != "" {
				results = append(results, ocrPageResult{Page: i + 1, How: "text"})
				b.Write([]byte(layer[i] + "\n"))
				continue
			}
			// A genuinely blank page. Recorded as OCR'd, contributing nothing —
			// which is what makes "the engine read 12 pages and they are blank"
			// a different answer from "this box has no engine".
			results = append(results, ocrPageResult{Page: i + 1, How: "ocr"})
			continue
		}
		if text == "" {
			// The engine read it and there is nothing on it, and neither is there
			// a text layer to fall back to. Same answer as above, reached without
			// pretending a comparison happened.
			results = append(results, ocrPageResult{Page: i + 1, How: "ocr"})
			continue
		}
		results = append(results, ocrPageResult{Page: i + 1, How: "ocr"})
		b.Write([]byte(text + "\n\f"))
	}
	if more {
		// Not the real total, and never presented as one: this build knows
		// "more than the cap" and says exactly that.
		total = -1
	}
	return b.buf.String(), results, total, false, nil
}

// layerAt is page i's text layer, or "" when the layer pass returned fewer pages
// than the job is reading — which is every page on the fallback path, where
// there is no layer at all. A helper rather than an inline bounds check because
// the comparison above reads it twice and an off-by-one in one of the two reads
// would silently turn the comparison back off.
func layerAt(layer []string, i int) string {
	if i < 0 || i >= len(layer) {
		return ""
	}
	return layer[i]
}

// renderOnePage draws ONE page of the PDF into dir and returns the PNG's path.
//
// Its own output prefix per page, because pdftoppm pads the page number in the
// filename to the width of the DOCUMENT's page count — page 5 of a 12-page file
// is p5-05.png — so a shared prefix would need that width guessed. A prefix that
// is unique per page makes the glob exact whatever poppler decides to call it.
func renderOnePage(dir, in string, page int, budget time.Duration) (string, error) {
	prefix := filepath.Join(dir, fmt.Sprintf("pg%d", page))
	_, rerr := runCmd(budget, "pdftoppm", "-png", "-gray", "-r", ocrDPI,
		"-f", fmt.Sprint(page), "-l", fmt.Sprint(page), in, prefix)
	// A NON-ZERO EXIT IS NOT AUTOMATICALLY A FAILURE HERE. pdftoppm complains
	// about damaged objects on PDFs it nonetheless rendered, and refusing a page
	// poppler was willing to draw would be worse than reading it. What decides
	// is whether the page came out.
	got, gerr := filepath.Glob(prefix + "-*.png")
	if gerr != nil || len(got) == 0 {
		if rerr != nil {
			return "", fmt.Errorf("pdftoppm could not rasterize page %d: %w", page, rerr)
		}
		return "", fmt.Errorf("pdftoppm produced nothing for page %d", page)
	}
	sort.Strings(got)
	return got[0], nil
}

// renderPageProbe is the FALLBACK ordering, used only when the layer pass
// answered with nothing: render up to the cap plus one page, and take the page
// count from what came out.
//
// This is what pdf-ocr used to do on every document. It survives here because
// the layer pass carries BOTH the text and the page count, so a document poppler
// will not read the text of leaves the job with no count either — and a page
// count is not optional, since "there is more of your document past page 30" is
// something the reply has to be able to say. Rendering is the only other way to
// obtain it without a third binary.
//
// Returns (nil, false, nil) when the deadline is already gone, which the caller
// reports as a timeout rather than as an empty document.
func renderPageProbe(dir, in string, pageCap int, deadline time.Time) ([]string, bool, error) {
	budget := time.Until(deadline)
	if budget <= 0 {
		return nil, false, nil
	}
	prefix := filepath.Join(dir, "probe")
	_, rerr := runCmd(budget, "pdftoppm", "-png", "-gray", "-r", ocrDPI,
		"-f", "1", "-l", fmt.Sprint(pageCap+1), in, prefix)
	pages, gerr := filepath.Glob(prefix + "-*.png")
	if gerr != nil || len(pages) == 0 {
		if rerr != nil {
			return nil, false, fmt.Errorf("pdftoppm could not rasterize this PDF: %w", rerr)
		}
		return nil, false, fmt.Errorf("pdftoppm produced no pages from this PDF")
	}
	sort.Strings(pages)
	if len(pages) > pageCap {
		// Remove the probe page before anything reads it: it exists only to
		// answer "is there a page 31", and OCRing it would return text from a
		// page this job is not claiming to have done.
		for _, extra := range pages[pageCap:] {
			_ = os.Remove(extra)
		}
		return pages[:pageCap], true, nil
	}
	return pages, false, nil
}

// pdfTextByPage is the PDF's own text layer, one entry per page, for the first
// n pages. A page with no layer is the empty string.
//
// THE SECOND RETURN IS LOAD-BEARING and is why this is not just a nil slice.
// Since the layer pass runs first, it is also what the page count comes from, so
// "poppler answered, and these pages have no text" and "poppler did not answer"
// are two different situations with two different next steps: the first OCRs
// every page, the second has no idea how many pages there are and has to fall
// back to rendering to find out. Collapsing them would silently report a
// document poppler refused as a zero-page document.
//
// BOUNDED AS IT ARRIVES, via runExtract rather than runCmd. This is the first
// thing every pdf-ocr job now runs, and a measured 3.2 MB PDF whose
// 20,000 pages share one compressed stream making pdftotext emit ~83 MB. A
// capped or timed-out read is not a page count either, so both count as "did not
// answer".
func pdfTextByPage(in string, n int, deadline time.Time) ([]string, bool) {
	budget := time.Until(deadline)
	if budget <= 0 {
		return nil, false
	}
	out, timedOut, err := runExtract(budget, "pdftotext", "-layout", "-q",
		"-f", "1", "-l", fmt.Sprint(n), "--", in, "-")
	if err != nil || timedOut || len(out) > ctxExtractMaxText {
		return nil, false
	}
	// pdftotext separates pages with a form feed and emits one after the last
	// page, so the split has a trailing empty element to drop.
	parts := strings.Split(out, "\f")
	if len(parts) > 0 && strings.TrimSpace(parts[len(parts)-1]) == "" {
		parts = parts[:len(parts)-1]
	}
	if len(parts) == 0 {
		// Not one form feed, so not one page: poppler produced no output at all.
		return nil, false
	}
	return parts, true
}