Blog Details

A clinical operations team needed hundreds of formatted lab reports annually. Each report was built from a PDF provided by a third-party laboratory. The initial request seemed simple: extract tables from PDFs, reformat them, and drop them into a template. No one asked for machine learning, and for the first few vendors, a script sufficed.

Trouble began when the pipeline encountered real-world conditions: a dozen labs, each with unique report generators, frequently changed table layouts without notification. Sample IDs varied, appearing clearly, truncated, or simply incorrect on the page. To create an unattended system producing trustworthy clinical reports, we realized extraction wasn't just a parsing problem. It was primarily a validation problem, with parsing as merely the initial, least reliable step.

The Illusion of a Simple PDF-to-Table Job

The first PDFs we looked at made the job seem trivial. Clean tables, consistent column headers, sample IDs sitting in an obvious cell at the top of the page. A standard table-extraction library handled maybe eighty percent of documents correctly on the first pass, which is exactly the number that makes a team confident enough to greenlight full automation.

That confidence didn't survive the second vendor. Their PDFs used merged cells for multi-analyte rows, put units in a footnote instead of the header, and occasionally split a single logical table across two pages with no repeated header row. A parser tuned to vendor one's structure silently mis-assigned values from vendor two, and because the output still looked like a plausible table, nobody caught it until a clinician noticed a result that didn't match the patient's history.

That near-miss reframed the whole project. The question stopped being 'can we extract this table' and became 'how do we know when we can't trust what we extracted,' which turned out to be the more useful question by a wide margin.

Template Matching Per Vendor: Our First, Doomed Approach

Our next attempt was template matching: for each lab, hand-build a layout profile describing where the sample ID lived, which columns held which analytes, and what row patterns marked the start and end of a results table. This is a reasonable approach when you have three or four vendors and stable layouts, and it worked well enough to get the pipeline into production for the first wave of labs.

It broke down as the vendor list grew. Labs update their reporting software on their own schedule, and a font change or a new disclaimer paragraph could shift table coordinates just enough to throw off a coordinate-based template. We were maintaining upwards of fifteen templates within a year, and every silent layout change turned into a support ticket days or weeks later, once someone downstream noticed a malformed report.

The deeper issue was that templates encoded assumptions about position and formatting rather than about meaning. A template could tell you where the sample ID usually sits; it couldn't tell you that the string sitting there was actually a sample ID versus a batch number the lab had started printing in the same location after a software update.

“A pipeline that's right ninety-five percent of the time and silent about the other five percent is more dangerous than one that's honest about its uncertainty.”

Why Reading Sample IDs Literally Falls Apart

Sample IDs turned out to be the single biggest source of downstream errors, and they're deceptively easy to underestimate because they look like plain text sitting right there on the page. In practice, we saw plenty of ways an ID could go wrong. Some were OCR-mangled by a low-resolution scan, some got wrapped across two lines and split into what looked like two separate values, and some were flat-out transcribed incorrectly by the lab's own system, not matching the ID in the originating order.

Reading the ID verbatim and trusting it meant propagating any of those errors straight into a formatted clinical report, which is the worst possible place for an ID mismatch to surface. We tried a stricter OCR pass with post-processing corrections for common character confusions, and it reduced but didn't eliminate the problem, because some of the errors originated in the source document itself, not in our extraction of it.

That was the moment we accepted that literal extraction of the sample ID was the wrong strategy entirely. The ID printed on the page needed to be treated as one signal among several, not as ground truth, and the pipeline needed a way to predict the correct ID and flag disagreement rather than blindly trust whatever string sat in the expected cell.

Sample-ID Prediction Heuristics That Actually Held Up

The heuristic that ended up working reframed sample ID resolution as a small evidence-combination problem rather than a text-extraction problem. Instead of reading one field and trusting it, the pipeline gathered every available signal on the page and in the surrounding order metadata, then predicted the most likely correct ID and scored its own confidence.

The single biggest gain came from cross-referencing the order system's expected ID format and sequence against whatever the PDF contained, since lab IDs typically follow a predictable structure per client and per date range. A string that broke that structure was treated as suspect even if OCR reported high confidence for the characters themselves. Combining that structural check with fuzzy matching against the batch of IDs expected for that day's run closed most of the remaining gap.

We also learned to weight the ID's position by how much the rest of the layout had already deviated from what the pipeline expected. A lab that had changed one thing about its report had usually changed more than one.

• Cross-check extracted ID format against the known ID pattern for that lab and date range
• Fuzzy-match candidate IDs against the batch of IDs expected from that day's order queue
• Down-weight OCR confidence when the surrounding layout has already deviated from the expected template
• Treat multiple partial matches (e.g., split across two lines) as a single candidate before scoring
• Always emit a confidence score alongside the predicted ID, never just the ID itself

Validation Checkpoints: The Gatekeepers Between Extraction and Delivery

Prediction heuristics only earn their keep if there's somewhere for low-confidence output to go besides straight into a finished report. We built the pipeline around a series of validation checkpoints, each one a small, specific gate that a document had to pass before moving to the next stage, with automatic routing to a human review queue for anything that failed.

The checkpoints were deliberately narrow rather than one big 'does this look right' check, because narrow checks are easier to reason about, easier to test, and produce actionable error messages instead of a vague failure. A table-shape checkpoint verified expected row and column counts before a value-range checkpoint even ran, so a completely misread table never got compared against clinical thresholds and generated a confusing false alarm.

This staged structure is also what made the pipeline safe to run unattended. Every checkpoint that passed added confidence; every checkpoint that failed stopped the document cold and logged exactly why, which meant the human reviewers spent their time on genuinely ambiguous cases instead of re-deriving what had gone wrong from scratch.

Chart Annotation and What Running Unattended for a Year Taught Us

Once tables were reliably extracted and validated, the reports still needed trend charts annotating notable results against reference ranges. That step introduced its own failure mode: a chart can be visually plausible while pointing at the wrong data point, especially when a patient has multiple samples on file with similar values. We handled this by tying every annotation directly to the same validated row that produced it, rather than letting the charting step re-derive position from raw coordinates, and by adding a checkpoint that confirmed chart-plotted values matched the underlying table before a report was finalized.

Over roughly a year of unattended operation across several hundred reports, the checkpoint system caught issues on a meaningful minority of documents, almost always concentrated around vendors who had recently changed their report generator or scanned documents at unusually low resolution. That concentration was itself useful signal, because it told us where to invest in better heuristics next rather than treating every failure as equally worth chasing.

This holds well beyond one pipeline. In clinical automation, the accuracy of any single extraction step matters less than the reliability of the system's ability to know when that step failed. A pipeline that's right ninety-five percent of the time and silent about the other five percent is more dangerous than one that's right ninety percent of the time and honest about its uncertainty. Clinicians can act on flagged uncertainty, but they can't act on an error they never see.

• Tie chart annotations directly to validated table rows, never to re-derived coordinates
• Add a checkpoint comparing plotted chart values against source table values before finalizing
• Track failures by vendor and cause to prioritize heuristic improvements
• Treat unattended reliability, not raw extraction accuracy, as the north star metric