Turning long, ugly PDFs into structured data
An asynchronous extraction service for a maritime software platform. Long technical documents go in; structured blocks, markdown and search-ready chunks come out. The customer's file is read once and never stored.
- Caller's bucketthe file stays theirs
- APIstateless, no bytes
- Queuejob id only
- GPU workerone job at a time
- Result storeexpires in days
- Poll or webhooknever the only path
- Technical manuals, drawings and scans defeat naive PDF-to-text. Tables break across pages, drawings hold text a layout model never sees, and scanned pages need OCR.
- Several teams needed the same capability behind one stable contract, without each of them owning a GPU pipeline.
- One GPU instance to start, and jobs must survive a restart.
- The PDF-parsing stack leaks memory without bound on long documents.
- The vision model is non-deterministic, even at temperature zero.
- Customer files must not be retained.
Decisions.
Async only, one code path.
A synchronous mode is the first thing every caller asks for and the first thing to break: the client times out, the GPU keeps working, and nobody collects the result. So there is no synchronous endpoint at all.
We never store the input.
Callers keep the PDF in their own bucket and grant a role to read it. Each tenant carries a bucket allowlist, otherwise one tenant could name another's bucket and read it through us. The working copy lives on disposable scratch disk and is deleted in a finally block, backed by a janitor.
A lease on the row, not a supervisor.
Claiming a job stamps an owner token and an expiry in one conditional update. A crashed worker's job becomes claimable when the lease lapses, a live worker can't be robbed, and a stale worker's writes match nothing. Extending queue visibility is the liveness signal, so there is no heartbeat table to maintain.
End the process to free the memory.
The parser leaks, and only ending the process reclaims it. Documents are cut into short slices, each in its own child process. A long document that failed every attempt as one process passed every attempt as several. Pages carry absolute numbers, so merging follows the plan, never the completion order.
One contract, generated on both sides.
The TypeScript API and the Python worker generate their types from a shared schema, and CI regenerates them and fails on any difference. The schema has exactly one owner.
Escalate per page; prefer missing to wrong.
Pages go to a vision model only when local checks fail. A truncated or filtered AI answer is thrown away and the local text kept. A failed slice leaves a marked hole, never a silent gap, and “partial” is a first-class result.
- Five job states, including partial, with a machine-readable list of what not to trust.
- Routing thresholds come from a labelled evaluation, not from the parser's own confidence scores, which turned out not to separate good pages from bad.
- Cost and per-page provenance are recorded on every job.
- One routing threshold is still a placeholder until it is measured on the target hardware. I'd rather say so than dress it up.