GNN chunking plus adapters (pandas, etc.). Oracle runs are listed separately below.
| # | Method | Chunks | R@1 | R@5 | R@10 | MRR | sR@1 | sR@5 | sR@10 | sMRR | Judge Avg | Judge ≥4 |
|---|
Same queries as above; chunkers run on the per-sheet JSON under
rag_eval_cache (else data/labeled) without running the GNN.
The default cache from downloading sheets is usually not human-labelled for table
structure—so this is not automatically a “gold structure” ceiling. It only measures
human-quality chunking if you place labelled grids there for the same
sheet_ids as your RAG eval (same Q&A file; separate need for structure labels
on those eval sheets). Training labels alone do not help unless exported for those eval IDs.
Rows: GNN chunking runs (graph_* / graph_row_*) with human labels on held-out sheets.
Inference timing is wall-clock predict_sheet per sheet (graph build + forward + post); ms/node uses non-empty value cells as graph nodes (same as the GNN).
Oracle RAG scores (human chunking) measure a ceiling that still depends on retrieval, generation, and Q&A difficulty — not only perfect structure.
| Method | L4 chunk F1 | L4 soft F1 | L2 macro F1 | Judge | R@1 | sR@1 | mean ms/node | p95 ms/node | predict total s | nodes/s |
|---|
Exploratory only when n is small.