Cleaning text copied from PDFs and OCR
PDFs preserve a page’s appearance, not necessarily a clean reading order. OCR adds another interpretation layer. The result can look correct while carrying broken lines, invisible separators, substituted glyphs, and spacing that fails elsewhere.
Key takeaways
- Treat PDF and OCR text as extracted data, not as a reliable plain-text source.
- Review line joins, soft hyphens, whitespace, and visually similar substitutions separately.
- Compare important values with the source page after normalization.
Why PDF text breaks at page boundaries
A PDF stores positioned glyphs and layout instructions. Copying a paragraph may preserve hard line breaks from a narrow column, insert spaces between visual fragments, or merge text in an unexpected reading order. Headers, footers, and page numbers can also enter the selection.
Start by rebuilding paragraph boundaries. Join lines that were broken only by layout, preserve real paragraph breaks, and inspect words split at the end of a line. A discretionary soft hyphen may be invisible until search or comparison fails.
- Remove repeated headers, footers, and page numbers before editing the body.
- Distinguish layout line breaks from actual paragraph breaks.
- Check words split across lines before deleting every hyphen.
What OCR gets plausibly wrong
OCR errors are often believable: the letter O becomes zero, lowercase l becomes uppercase I, a dash becomes a minus sign, or a ligature becomes two unexpected characters. Automatic punctuation cleanup cannot decide whether an invoice number, surname, or measurement was recognized correctly.
Use character normalization for spacing and invisible marks, then compare names, dates, totals, identifiers, and quoted passages with the scanned page. Those fields deserve source verification because a tidy string can still contain the wrong visible character.
A reliable extraction handoff
Keep the original PDF open, clean a plain-text copy, and verify the high-value fields before moving the result into a CMS, spreadsheet, search index, or form. If the document is long, sample each layout type rather than checking only the first page.
Finish with a destination test. Search for a known phrase, verify paragraph wrapping, and confirm that identifiers match. The source comparison protects accuracy; the destination test protects compatibility.