Recently I ran into one of those "works on my machine" issues.
I built an app that runs on a Pi (which also works as my DNS server) to pull public data from the government and turn it into personal investment insights. The files are PDFs with tables.
It uses RAG methods to read reports and extract tables. I got the idea of mixing Camelot with a pdfplumber fallback thanks to Chandana Cheerla’s paper "Advancing Retrieval-Augmented Generation for Structured Enterprise and Internal Data". 🙏
On macOS it worked but on the Pi several rows were missing.
After checking, I found:
🧐 Camelot wasn’t installed right on macOS, so pdfplumber did the job.
🧐 On the Pi, Camelot was installed but only read part of each table. Since it didn’t fail, the fallback never ran.
The first fix was simple: compare the row count from both and keep the one with more data.
But that didn’t fix empty rows. So I added a score system that checks each row and gives points for filled fields, then picks the parser with the higher score.
Next step: merge both results and skip duplicate rows.
Another idea I saw is to render PDFs into JPEGs and use a vision LLM to read tables. A Nanonets post by Yeshwanth Reddy cited main issues as low repeatability and hallucinations. Raja Biswas and Bo Liu at NVIDIA also tested it and table accuracy was lower.
Right now, the top performer seems to be Azure Document Intelligence (Form Recognizer), which uses a small CNN (ResNet-18).
Jonathan Simon and James Williams at Elastic Search Labs showed strong results combining Azure’s output with their search models (full-text, sparse, dense, hybrid vector searches). Its main rival - Amazon Textract - is a bit faster, but less accurate.
A BusinessWare Tech blog named GPT-4o + OCR (using Azure Read or Tesseract) as the most accurate setup and about 20% cheaper too.
New papers appear almost daily with "better" methods, so it’s hard to tell what’s best right now.
PDF is incredibly "messy" because it was designed as a final presentation format to be rendered identically everywhere, not as a structured document model.
Roughly, a PDF is a collection of low-level objects. Text is stored as placement instructions - "draw character X at (x,y) using font Y" - and pages are a list of those instructions.
Some possible future directions are:
💡 A new PDF standard or wrapper: PDFs with embedded JSON/XML describing tables, fields, schema, etc.
- technically trivial but unlikely to happen.
💡 A hybrid PDF of human-readable plus a machine-readable version.
- already partly implemented via PDF/UA and PDF/A-3.
💡 Publish directly to CSV/JSON when tables are the primary content, and reserving PDF for human-readable versions.
- vastly simpler and cheaper, very likely.
💡 Increase use of vision/LLM pipelines to treat existing PDFs as images and extract semantic structure (fallback rather than format change).
- seems to be the trend, but with varied accuracy and hallucinations.
