Almost every production RAG pipeline does some version of the same thing: before embedding a chunk, prepend its heading path — document title, section trail, a breadcrumb. It costs nothing, it feels obviously right, and essentially nobody has tested whether the gain comes from the *information* in the breadcrumb or from the mere presence of extra tokens that make chunks look more like documents.
A paper posted to arXiv on October 7 (2610.10170, Kuehlkamp, Correa Saboia Moreira, and Rund) takes that question seriously enough to borrow methodology from clinical trials. The authors describe it as, to their knowledge, the first placebo-controlled measurement of structure-metadata injection: four pre-registered contrasts, scored with a coverage-aware nDCG, tested via document-clustered bootstrap with Holm correction, across two deliberately dissimilar corpora — 200 Wikipedia Featured Articles with 951 queries, and 1,585 QASPER papers with 4,303 questions. Their headline is that organization does help, but the design is built to separate *why* from *whether*.
Why the methodology is the story
Look at what's standard in this literature and the contrast is unflattering. Most chunking ablations report a single nDCG delta on one corpus, no significance testing, no correction for multiple comparisons, and — the quiet killer — treat chunks as independent samples when chunks drawn from the same document obviously are not. Document-clustered bootstrap exists precisely because that assumption inflates confidence. Pre-registration exists because it's trivially easy to try six structure treatments and write up the one that won.
The consequence is a field that contradicts itself. A study of oil and gas enterprise documents (arXiv 2603.24556) found structure-aware chunking consistently best on top-K metrics and called structure preservation essential in specialized domains. Work on semantic chunking has repeatedly found its compute cost unjustified against fixed-length baselines. A Vectara study at NAACL 2025 reportedly swept 25 chunking configurations against 48 embedding models and concluded chunking choice rivaled embedding choice in impact. These can't all generalize. Without placebo arms, you can't tell which results are mechanism and which are artifact.
What to actually do with this
If you run retrieval in production, the cheap move is to add a placebo arm to your own eval. Build a parallel index where the breadcrumb is syntactically identical but semantically wrong — scrambled headings, headings from a different document — and rerun. If your structure gains survive, they're real and you've earned the complexity. If they collapse, you've been paying indexing and token cost for a formatting effect, and you can spend that budget on reranking instead.
This lands against a parallel shift: agent memory is pulling away from RAG as a distinct subfield, with recent work probing relevance-versus-sufficiency gaps, retrieval admissibility, and governance as first-class failure modes. Memory systems are inheriting RAG's evaluation habits before those habits have been validated. Better to import the controls now than to rediscover this in two years with a new vocabulary.