The proof, in full
Everything on the main page traces to the tables below. Basis for all figures: frozen data snapshot of 2026-07-08, answers digit-verified against machine-computed ground truth, incumbent production stack measured live on the same data.
1. The benchmark
456 questions over a multi-domain corpus (7.05M records, 3 domains, 2024-2026), spanning six operation types. Every question's true answer was computed by machine from the frozen snapshot; NCL's answer was compared digit for digit. Result: 454 of 456 exact, 99.6%.
| Operation | What it demands | Exact | N | Rate |
|---|---|---|---|---|
| Aggregate | Sum a measure across a filtered window | 289 | 290 | 99.7% |
| Point value | Retrieve one exact cell for one entity and date | 68 | 68 | 100% |
| Share | Compute one entity's share of a total | 54 | 54 | 100% |
| Delta | Compare two windows and quantify the change | 28 | 28 | 100% |
| Funnel walk | Traverse sequential stages with correct stage arithmetic | 9 | 10 | 90% |
| Trajectory | Characterize a series over time | 6 | 6 | 100% |
| Total | 454 | 456 | 99.6% |
| Measurement | Value | Note |
|---|---|---|
| Context selection, median | 6.4 ms | question to complete relevant cell surface, from inside the model |
| Full answer, deterministic path | ~10 ms | selection + deterministic computation of the figure |
| Full narrated answer | 7-10 s | language model narrates over precomputed, certified figures |
| Incumbent comparable path | ~63 s | measured live on the same corpus |
| Selection recall | 0.9998 | near-complete data surface per question |
| Model size | 2.7M params | trained on a consumer laptop |
| LLM cost, deterministic path | $0 | no tokens spent producing figures |
2. Parity validation, and what it surfaced
Before any benchmark ran, NCL's source data was validated cell by cell against the incumbent production stack on a locked one-month slice across all 3 domains. The validation standard was exact parity: macro totals exact, daily sums exact, funnel stages within a fraction of a percent.
The incumbent is not named here by design: it is a live production system belonging to the data owner, and this document circulates outside that perimeter. Anonymizing it is confidentiality discipline, not convenience; it was measured live and every run is recorded.
The validation was strict enough to find 3 confirmed production defects in the incumbent itself, including a duplicated partial-day load that silently inflated one domain's totals. All 3 were documented with evidence. A benchmark whose data audit catches real bugs in the reference system is a benchmark whose numbers can be trusted.
3. Methodology, the bench laws
| # | Law | Why it matters |
|---|---|---|
| 1 | Frozen snapshot | All measurements run against one locked snapshot (2026-07-08). No moving target, fully reproducible. |
| 2 | Machine-verified pairs | Ground truth is computed from the data by machine: 4,767 question/answer pairs over 2.23M cells, zero human labeling, zero human error floor. |
| 3 | Incumbent measured live | The ~63 s baseline is the real production system answering on the same data, not an estimate. |
| 4 | Exact-match scoring | An answer counts only if it matches the computed truth digit for digit. No partial credit, no semantic grading. |
| 5 | The model never does math | Every figure is computed by a deterministic arm from NCL-selected cells; the language model parses and narrates only. Exactness is architectural. |
| 6 | Defects documented | Every anomaly found during validation, on either side, was recorded with evidence. 3 confirmed, all incumbent-side. |
4. The academic spine
548 records reviewed across Scopus and Web of Science. Every paper below was verified against its source; zero invented citations. The three-claim structure: the problem is real and documented (1), no alternative has closed it (2), the mechanism is proven feasible piece by piece (3), and no published system ships the full synthesis.
Claim 1. RAG is breaking in production
- Barnett et al., Seven Failure Points When Engineering a RAG System, CAIN 2024, IEEE/ACM. arXiv:2401.05856 indexed
- Chen et al., Benchmarking Large Language Models in Retrieval-Augmented Generation, AAAI 2024. arXiv:2309.01431, 522 citations indexed
- Cuconasu et al., The Power of Noise: Redefining Retrieval for RAG Systems, SIGIR 2024. arXiv:2401.14887 indexed
- Yu et al., Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Models, EMNLP 2024. arXiv:2311.09210 indexed
- Huang & Huang, A Survey on Retrieval-Augmented Text Generation for Large Language Models, ACM Computing Surveys. arXiv:2404.10981 indexed
- Fan et al., A Survey on RAG Meeting LLMs, KDD 2024. arXiv:2405.06211, 717 citations indexed
- Asai et al., SELF-RAG: Learning to Retrieve, Generate, and Critique, ICLR 2024. arXiv:2310.11511, 495 citations indexed
- Chan et al., Don't Do RAG: When Cache-Augmented Generation is All You Need, WWW 2025 companion. arXiv:2412.15605 proceedings
Claim 2. No alternative has closed the gap
- Liu et al., Lost in the Middle: How Language Models Use Long Contexts, TACL vol. 12, 2024. arXiv:2307.03172, 1,249 citations indexed
- Hsieh et al., RULER: What is the Real Context Size of Your Long-Context Language Models?, COLM 2024. arXiv:2404.06654 proceedings
- Modarressi et al., NoLiMa: Long-Context Evaluation Beyond Literal Matching, ICML 2025. arXiv:2502.05167 indexed
- Shi et al. (Google), Large Language Models Can Be Easily Distracted by Irrelevant Context, ICML 2023. arXiv:2302.00093 indexed
- Li et al. (Google), RAG or Long-Context LLMs? A Comprehensive Study and Hybrid Approach, EMNLP 2024 Industry. arXiv:2407.16833 indexed
- Lee et al. (DeepMind), Can Long-Context Language Models Subsume Retrieval, RAG, SQL and More? (LOFT), 2024. arXiv:2406.13121 preprint
- Yu et al., In Defense of RAG in the Era of Long-Context Language Models, 2024. arXiv:2409.01666 preprint
Claim 3. Corpus-in-weights is proven feasible, piece by piece
- Tay et al. (Google), Transformer Memory as a Differentiable Search Index, NeurIPS 2022. arXiv:2202.06991, 227 citations indexed
- Mehta et al. (Google), DSI++: Updating Transformer Memory with New Documents, 2022. arXiv:2212.09744 preprint
- Borgeaud et al. (DeepMind), Improving Language Models by Retrieving from Trillions of Tokens (RETRO), ICML 2022. arXiv:2112.04426 indexed
- Wang et al. (Microsoft), A Neural Corpus Indexer for Document Retrieval, NeurIPS 2022. arXiv:2206.02743, 122 citations indexed
- Wang et al. (Microsoft), KBLaM: Knowledge Base Augmented Language Model, ICLR 2025. arXiv:2410.10450 indexed
- Cheng et al. (Microsoft), xRAG: Extreme Context Compression for RAG with One Token, 2024. arXiv:2405.13792 preprint
- Liu et al. (Microsoft), TAPEX: Table Pre-training via Learning a Neural SQL Executor, ICLR 2022. arXiv:2107.07653 indexed
- Berges et al. (Meta), Memory Layers at Scale, ICML 2025. arXiv:2412.09764 indexed
- Allen-Zhu & Li (Meta), Physics of Language Models: Part 3.1, Knowledge Storage and Extraction, 2023. arXiv:2309.14316 preprint
- Dai et al., Knowledge Neurons in Pretrained Transformers, ACL 2022. arXiv:2104.08696, 427 citations indexed
- Meng et al., Locating and Editing Factual Associations in GPT (ROME), NeurIPS 2022. arXiv:2202.05262, 1,186 citations indexed
- Meng et al., Mass-Editing Memory in a Transformer (MEMIT), ICLR 2023. arXiv:2210.07229 indexed
- Su et al., Parametric Retrieval Augmented Generation, SIGIR 2025. arXiv:2501.15915 indexed
- Tan et al., Dynamic Parametric RAG (DyPRAG), 2025. arXiv:2503.23895 preprint
The boundary with the closest published work
Parametric RAG (Su et al., SIGIR 2025) is the nearest published system: it encodes individual documents into LoRA adapters and merges them into the LLM at query time, for open-domain QA. It is peer-reviewed proof that documents-into-parameters works, and it is exactly where the published frontier stops. NCL goes where it does not: an entire multi-domain structured corpus made native as the primary answer path, cold-started from computed ground truth with zero human labels, paired with a deterministic exactness arm, running fully inside the data perimeter, and benchmarked at 99.6% exact against machine-verified truth. The pieces are published. The synthesis is shipping here.
5. The fastest independent check
Every figure on this page reproduces from the frozen snapshot through the recorded benchmark harness. And because the pipeline is corpus-agnostic, machine-generated ground truth, no human labels, laptop-scale training, the strongest verification available to a skeptical reader is not this document: it is a live run of the same pipeline on data the reader supplies, answers checked against the reader's own numbers. This document is built to survive scrutiny; the system is built to survive that test.
All figures measured on the frozen 2026-07-08 snapshot. Recorded run outputs back every number on this page.
← Back to the pitch