Same open model, same interpolation question, different OCR method to build the card.
How much of the pass-rate gap is the model, and how much is the table extractor?

Each cell shows correct/total across the 12 interp questions for that (model, OCR method) pair. Cells are tinted by correct-rate (red → amber → green). Local open models can only read text, so they see the OCR-generated card — not the page image.

What to look for. Variance going across a row tells you how much the OCR method matters for that particular model. Variance going down a column tells you how the same OCR method serves different open models. If a column is roughly green across all three models, that OCR method is producing usable cards for the open-tier. If a row is consistently green, that open model is robust to OCR noise. Qwen 2.5 VL 7B is the first open multimodal model in the grid — it only fills the "direct vision" column (it sees the raw page image instead of a text card).

Same 12 questions, but answered by closed-tier flagship APIs. The first 5 columns are the same OCR-text setup as above — flagship reads the per-OCR card, no image. The rightmost column No OCR — direct vision skips the OCR step entirely: the flagship sees the raw page image plus the question and answers in one shot.

The question this answers. If the "direct vision" column is the greenest, the OCR pipeline is adding more noise than signal even for SOTA models — they'd be better off just looking at the page. If the best OCR column beats direct vision, OCR is structurally useful even for the top of the stack (and the cheapest OCR that hits that ceiling wins on cost).

Both Grok-4 and Qwen 2.5 7B were tested against every Model Evals card variant + several floor cases (raw EasyOCR with no Docling layout, plain PDF text layer, direct page vision). Same 12 interp questions for each cell. Sorted by Qwen 2.5 7B correct count — the interesting axis is which formats lift the open-tier model above the prior 9/12 ceiling.

What this shows. Grok-4 is roughly format-insensitive — ~13 variants all score 12/12. Qwen 2.5 7B is much more format-sensitive: its peak (10/12 onmicro-1k, compact-4k, csv-plus-*, labeled,table-normalized) is one above its score on the project's defaultdocling-easyocr (9/12). Compact-with-context formats help the open model materially. Floor cases — no-frontmatter, easyocr-raw, gpt4o-vision — hurt both, but hurt the open model more.

Two axes: USD per page (commercial API list price; $0 for local tools) and seconds per page (wall-clock on the project's M-series Mac). The two are not interchangeable — a free local tool can be the slowest of the bunch, and the cheapest paid API can be the fastest.