agentic.dev

AI Data Extraction: PDFs, Invoices and Unstructured Documents

Published 2026-02-21

AI Data Extraction: PDFs, Invoices and Unstructured Documents

Every day, businesses process millions of PDFs, invoices, and unstructured documents-manually. According to recent estimates, employees spend up to 50% of their time on data entry and extraction tasks. The opportunity cost is staggering, especially when modern AI can automate most of this work with near-human accuracy.

In this guide, I’ll walk through practical techniques for extracting structured data from unstructured documents using state-of-the-art AI models. We’ll cover OCR, LLM-based parsing, and hybrid approaches-complete with code snippets and real-world tradeoffs.

---

Why Traditional OCR Falls Short

Optical Character Recognition (OCR) has been around for decades, but it’s not enough for modern document processing. Tools like Tesseract or Adobe Extract can convert scanned text into machine-readable format, but they fail at:

Layout understanding: OCR doesn’t distinguish between headers, tables, or footnotes.

Semantic parsing: Extracting key-value pairs (e.g., "Invoice Date: 2026-03-15") requires additional logic.

Handwritten text: Most OCR models struggle with non-printed text.

For example, here’s a common issue when using PyTesseract on an invoice:

import pytesseract
from PIL import Image
text = pytesseract.image_to_string(Image.open('invoice.png')) print(text)  # Output: "Invoice\nDate 03/15/2026\nTotal $1,200.50"

While this extracts raw text, you still need regex or manual rules to parse "Date" and "Total" into structured fields. This approach breaks if the invoice template changes.

---

Modern AI Document Parsing Techniques

1. LLM-Powered Extraction

Large Language Models (GPT-4o, Claude 3, etc.) excel at understanding document context. Instead of brittle regex, you can prompt an LLM to extract structured JSON from raw text:

from openai import OpenAI
client = OpenAI()
def extract_invoice_data(text):     response = client.chat.completions.create(         model="gpt-4o",         messages=[             {"role": "system", "content": "Extract invoice data as JSON."},             {"role": "user", "content": text}         ],         response_format={"type": "json_object"}     )     return response.choices[0].message.content
Example output:
{"vendor": "Acme Corp", "date": "2026-03-15", "total_amount": 1200.50}

Pros

Handles varied templates without retraining.
Understands synonyms (e.g., "Total" vs. "Amount Due").

Cons

Cost (~$1 per 1K pages at GPT-4o prices).
Slower than traditional OCR (500ms+ per page).

2. Specialized Document AI Models

Google’s Document AI, AWS Textract, and open-source alternatives like Donut (Transformer-based) are fine-tuned for documents:

# Using Donut (Hugging Face)
from transformers import DonutProcessor, VisionEncoderDecoderModel
import torch
processor = DonutProcessor.from_pretrained("naver-clova-ix/donut-base-finetuned-docvqa") model = VisionEncoderDecoderModel.from_pretrained("naver-clova-ix/donut-base-finetuned-docvqa")
Process image and generate JSON
pixel_values = processor(Image.open("invoice.png"), return_tensors="pt").pixel_values
outputs = model.generate(pixel_values, max_new_tokens=256)
decoded = processor.batch_decode(outputs)[0]

Pros

Faster than LLMs (~100ms latency).
Better at handwritten text.

Cons

Requires fine-tuning for custom document types.

---

Hybrid Approach: OCR + LLM

For high-volume processing, combine OCR for text extraction and LLMs for parsing:

First pass: Use Tesseract/Textract to extract raw text.

Second pass: Feed text into a lightweight LLM (e.g., Mistral 7B) for structuring.

# Pseudocode for hybrid pipeline
raw_text = pytesseract.image_to_string(image)
structured_data = query_llm(f"Extract fields from: {raw_text}")

Tradeoffs

10x cheaper than pure GPT-4o.
Still struggles with complex tables.

---

Handling Edge Cases

1. Tables in PDFs

Tabular data requires specialized tools like Camelot or PDFPlumber:

import pdfplumber
with pdfplumber.open("report.pdf") as pdf:     for page in pdf.pages:         table = page.extract_table()         # Convert to pandas DataFrame

2. Handwritten Fields

Fine-tune Donut or use Google’s Document AI Handwriting model.

3. Multi-Page Documents

Chunk documents and process sequentially, then merge results with an LLM:

# Split and process pages in parallel
pages = [pdf.pages[i] for i in range(10)]
results = Parallel(n_jobs=4)(process_page(page) for page in pages)
final_json = merge_results_via_llm(results)

---

Key Takeaways

Pure OCR isn’t enough: Use LLMs or specialized models for semantic understanding.

Hybrid pipelines save costs: Combine OCR for text extraction with small LLMs for parsing.

Test edge cases: Handwriting, tables, and multi-page docs need special handling.

For high-scale deployments, consider serverless document AI pipelines (AWS Textract + Lambda) or open-source self-hosted options (Donut).

--- Published on [agentic.dev](https://devaa-io.github.io/agentic-dev/)