Relevance Is Not Consequence: A Pre-Registered Falsification of Graph-Based Consequence Retrieval for AI Memory

Abstract

We report the pre-registered falsification of a memory-retrieval thesis and the fourteen-month research program that produced it. The thesis — relevance is consequence, not similarity — held that an AI memory system whose stored items are connected by typed causal edges, and whose retrieval traverses those edges, would surface more useful memories than embedding-similarity search, particularly for questions about causes and effects. We built the system (ngramDB: sparse distributed encoding, LLM-assisted typed causal extraction at ingestion, spreading-activation retrieval with no language model at query time) and subjected the thesis to a sequence of experiments whose pass/fail criteria were frozen before any result existed. The thesis failed every instrument. On a public long-conversation benchmark, embedding similarity achieved 86.0% recall@200 to the typed graph's 77.6%. On a self-built consequence-question probe over thirteen real incident postmortems and architecture decision records — conditions chosen to maximally favor the graph — similarity ranked the author-written root-cause passage at median rank 28 of 657 passages; the graph, at rank 77, winning 1 of 8 paired comparisons. Ablating all 1,195 causal edges moved the graph's median from 77 to 74: the causal structure, though genuinely extracted (over a thousand typed relations, with edges touching the true root-cause passages in 6 of 8 questions), contributed approximately nothing to ranking. A sharper finding cut both ways: on every consequence question, both mechanisms ranked a topically adjacent same-document passage above the actual causal answer — the question class is real, poorly served by both mechanisms we tested, and this implementation's graph traversal is not the answer to it. A post-publication audit of the frozen graph (§5.1) separates the failure mechanically: the pinned configuration seeded a majority of the corpus per query, so the graph arm re-ranked rather than traversed, and three of eight gold passages had no causal path reaching them at all. A subsequent baseline characterization (§5.2) found the decoy failure universal: BM25, hybrid fusion, a strong cross-encoder, and an LLM-reranker ceiling all preferred a same-document topical decoy to the gold passage on all eight questions — no mechanism tested achieves recall@1 above zero on this instrument. Five pre-registered falsification clauses fired during the program; all five were honored as written. We describe the system, the experiments, the discipline that kept a solo research program honest, what survived measurement (a deterministic, provenance-complete substrate; extraction-for-explanation rather than extraction-for-ranking), and the narrower successor hypotheses now under pre-registration. We argue that publishing this result is worth more than the thesis was.

Keywords: negative results, retrieval, AI memory, knowledge graphs, causal extraction, pre-registration, spreading activation, embedding similarity

1. Introduction

Biologically inspired memory is an attractive design language. Human memory does not run nearest-neighbor search over an embedding table; it consolidates, associates, strengthens with use, and — on one influential reading — organizes itself around what follows from what. If an engineered memory could retrieve by consequence, the argument went, it would answer the questions practitioners actually ask of institutional memory: not "what text resembles this text," but "what caused this," "what happened last time," "what should I expect if I do X."

We committed to that thesis in 2025 and spent fourteen months building and testing it. This paper reports that the thesis, in its retrieval form, is false as far as our instruments can see — and that we believe the instruments, because we designed them to be hard to argue with: pre-registered pass/fail criteria frozen before results existed, falsification clauses applied verbatim when they fired, ablations that isolate the causal structure's contribution, and a final experiment run on real documents under conditions deliberately favorable to the hypothesis.

Negative results in applied AI are chronically under-reported, and the specific claim we tested — that graph structure over memories improves retrieval relative to dense similarity — is a live commercial claim across the industry. We are not in a position to falsify anyone else's system. We are in a position to report, with unusual completeness, what happened when one team tried very hard to make the claim true for its own system, measured honestly, and lost. The methodological contribution — a pre-registration discipline workable for a solo industry researcher — may outlast the empirical one.

This program was conducted by a single human researcher working with AI assistants throughout; the division of labor, and the controls that keep AI assistance out of the measurement loop, are described in §3 and disclosed formally in the AI Assistance Disclosure section.

2. The System Under Test

ngramDB is a purpose-built memory store. The properties relevant to this paper:

Encoding. Text chunks ("engrams") are stored with sparse distributed representations for content addressing and full source text preserved immutably.

Extraction at ingestion. A two-pass pipeline: deterministic candidate extraction (Pass 1), then LLM-assisted canonicalization and typed relation extraction (Pass 2) producing subject–predicate–object causal relations with provenance. The final experiment (§5) ran extraction on deepseek-ai/DeepSeek-V4-Pro served via Nebius AI Studio; the formation lineage (§4.4) ran on gemma4:31b via Ollama Cloud. On the final experimental corpus, Pass 2 considered 1,021 relations and the concept→engram bridge projected 797 edge writes (549 distinct stored concept_relation edges after collisions and later overwrites); an additional materialization step created 646 consequence edges from stored causal triples. 32.1% of eligible relation objects were mentioned by two or more engrams — the precondition for cross-passage edges. 15.2% of engrams (100 of 657) were quarantined by extraction-quality gates and retained Pass-1 quality only.

Graph. Edge types include lexical co-reference (IDF-weighted keyword overlap), SDR similarity, within-document structural edges, and the causal classes above. Storage is one weighted slot per ordered node pair; retrieval-time propagation reads scalar weights and is blind to edge type. Type-aware alternatives were tested and closed earlier in the program (§4.4); a query-intent-conditioned, direction-aware traversal was designed but never built, and §9 scopes the conclusion accordingly.

Retrieval. Spreading activation from an SDR-encoded query over the weighted graph, deterministic, with no language model at query time — an invariant we held throughout the program and re-examine in §7.

Baselines. A pinned local embedding model (snowflake-arctic-embed2) with cosine ranking over the identical engrams; on the public benchmark, the harness's frozen dense-retrieval baseline.

3. Methodology: Pre-Registration for a Solo Program

Every experiment in this paper follows the same contract, adopted after early failures (§4.1–4.2) taught us that our own optimism was the main threat to validity:

1. Falsification clauses frozen blind. The claim, the instrument, the metric, and the pass/fail thresholds are written and ratified before any result exists. Thresholds are never adjusted after a number is seen; when a clause is ambiguous, the outcome is recorded as INDETERMINATE rather than resolved post hoc.
2. Report, don't tune. Weight or parameter tuning against a benchmark score is prohibited. Configuration comes from previously validated runs, with every deviation logged.
3. Artifacts are advisory; the runtime is authoritative. Trackers, summaries, and prompts have repeatedly diverged from ground truth; every load-bearing mechanic is re-verified against live source before it is relied upon, and instrument defects found this way (three during the final experiment alone: a fidelity bug in a replay harness, a path-dependent double-consolidation taint, and a manifest with no recorded build SHA) invalidated runs before their results could be read.
4. Deterministic instruments. Retrieval experiments are LLM-free and double-passed; a rank table that cannot be reproduced bit-identically is a stop condition, not a footnote.
5. Blind human adjudication where judgment is unavoidable. For the self-built probe (§5), gold labels were adjudicated by the operator before any retrieval ran, making blindness to condition structural rather than procedural.

Division of labor. The program was run by one human researcher (the author) with AI assistants in two distinct roles: a conversational LLM assistant for experiment design, specification authoring, source-code verification sweeps, results analysis, and manuscript drafting; and an agentic coding tool for execution — harness code, ingestion runs, scoring scripts, and bookkeeping — operating under written, pre-registered instructions. The human author made every ratification decision, froze every threshold, adjudicated every gold label, and performed every commit. Two controls keep AI assistance out of the measurement loop: all reported measurements are deterministic and LLM-free (no model served as judge, answerer, or scorer in any number in this paper), and every load-bearing mechanic asserted by an assistant was re-verified against live source before use — a rule adopted precisely because plausible-but-unverified assertion is the characteristic failure mode of AI-assisted research. That rule earned its keep: of the three instrument defects caught during the final experiment, some were introduced by AI-written harness code and some were caught by AI review of it; the discipline, not the assistant, is what made the result trustworthy in both directions.

Five falsification clauses fired across the program. All five dispositions were applied as pre-registered. We consider that record, more than any architecture, to be the paper's credential.

4. The Program: Four Prior Falsifications

We summarize the arc chronologically; each subsection is one claim, one clause, one disposition.

4.1 Enrichment pre-pass (closed). Early claim: LLM enrichment of source text before ingestion improves downstream retrieval. Result: one treatment corrupted source text; the other measurably added nothing. Disposition: closed; source text became immutable by construction thereafter.

4.2 Provenance self-report (closed). Claim: the extraction model can accurately self-report the provenance of its own outputs. Measured precision was approximately 20%. Disposition: closed; provenance became a structural property of the pipeline rather than a model claim.

4.3 Concept-level typed propagation on a public benchmark (closed). Claim: typed-graph traversal at the concept level improves retrieval on a public long-conversation benchmark. Result: end-to-end 73.12% vs. the harness's similarity baseline at 78.25%; recall@200 77.60% vs. 86.02%. Disposition: closed for that mechanism on that workload; the falsification was scoped to concept-level propagation, leaving the engram-level mechanism for §5.

4.4 Formation, and where it stopped mattering (closed at discrimination). Two experiments separated whether causal edges can form from text from whether they help. With hand-created, known-good edges, retrieval worked: the June baseline ranked the causal antecedent at mean rank 2.38 under graph activation versus 12.38 under similarity on a 27-engram corpus of eight probes engineered so similarity buries the cause (consequence-win 7/8, single run), and the later three-arm experiment's ceiling arm reproduced the effect on a second corpus at 0.875, identical across three replicates and robust to a 4× edge-weight clamp — the existence proof that kept the thesis alive. Text-extracted edges, by contrast, initially failed to form at all (a repeatable null across 24 attempts) until an identity-resolution primitive was built and validated blind, after which edges formed and traversal reached the causal antecedent (rank 30 → 6 on the diagnostic). Discrimination, however, remained at 0 of 8 on the synthetic corpus: with formation working, the system still could not prefer the causal answer over engineered decoys. Because the primary retrieval path is blind to edge type, three type-aware mechanisms were tested separately rather than left untried: static predicate weighting (boosting causal edge classes at write time) was pre-registered and closed after producing no discrimination gain; a causal-claim projection path (materializing extracted claims as their own retrievable structure) was pre-registered and closed the same way; and the concept-level typed propagation path — the one mechanism that traverses typed relations as typed relations — was tested on the public benchmark (§4.3) and again as an informational arm on the real corpus (§5), retrieving nothing in 8 of 8 questions both times. What was never built is a query-intent-conditioned traversal that treats causes, enables, and their inverses differently by direction and question type; the conclusion in §9 is scoped to exclude it. The bottleneck was named, in writing, as discrimination — not formation — before the final experiment was designed.

5. The Kill Test: Consequence Questions on Real Documents

Design. The final experiment was constructed as a kill test under conditions maximally favorable to the thesis: the most causally explicit genre in existence (eight real incident postmortems — Cloudflare, GitLab, AWS S3, Twilio, and four GitLab incident reviews — plus four architecture decision records, thirteen frozen documents, SHA-verified), author-written gold (each document's own root-cause or consequences passage), the production extraction model, and questions taken verbatim from each document's symptom/impact text. Construction rules were frozen before generation: a lexical-overlap filter (Jaccard < 0.15 between question and gold) excluded pairs embeddings solve trivially; the corpus itself served as the adversarial distractor pool; the operator adjudicated every pair blind to condition, with all selections, edits, and drop decisions recorded. Final probe: n = 8 pairs over 657 engrams. The instrument is self-built and published in full — a stated mitigation, not an apology.

Conditions. (a) Graph: engram-level spreading activation over the full edge set, one pinned request body, deterministic, double-passed. (b) Similarity: pinned-embedding cosine over identical engrams. (c) Ablation: condition (a)'s datadir with every causal edge (549 concept_relation + 646 consequence; 1,195 total) removed via the store's own primitive, survivor weights verified bit-identical. Secondary informational arms: a consolidation-cycle variant and the concept-level path from §4.3. Two closure rules were pre-registered separately: an extraction closure (if no causal edge touched any gold passage, the finding would be about edge formation, and no retrieval tier would be reported) and a retrieval closure (tiers bind only if the precondition passes). The precondition passed: causal edges touched the gold set in 6 of 8 questions.

Results.

Both pre-registered UNSUPPORTED conditions were independently satisfied; a sensitivity analysis excluding the two pairs nearest the overlap threshold does not change the outcome (1 of 6). The ablation is the decisive attribution: removing every causal edge slightly improved the graph's median. The extracted causal structure — real, gold-touching, and expensively obtained — contributed approximately nothing to ranking, and the graph arm's behavior is explained by its generic relatedness topology.

The finding underneath the tier. In 8 of 8 questions, under both primary arms, the best-ranked non-gold passage from the same document outranked the gold passage, usually at rank 1. Neither spreading activation nor embedding similarity isolates a causal antecedent from topical proximity; the winning baseline still placed the author's own root cause at median rank 28 with zero recall@1. Consequence questions are a retrieval class poorly served by both mechanisms we tested — whether stronger retrievers (hybrid sparse-dense, cross-encoders, relation-aware graph traversal) serve it is untested here. That is the one respect in which the founding intuition survives — as a diagnosis of a failure mode, not as our solution to it.

5.1 Post-Publication Path Audit (v0.3 addendum)

Prompted by external review — specifically the criticism that "a causal edge touches the gold passage" is too weak a precondition — we ran a pre-registered, descriptive audit of the frozen arm-(a) graph: for each probe, does a directed causal path exist from the query's activation seed set to the gold passage, and what does it look like? The audit was frozen as unable to reopen the retrieval tier in either direction, ran read-only on hash-verified copies with double-passed bit-identical output, and is released in full. Its first run executed against the wrong snapshot — the pre-materialization datadir — and was caught by its own edge-type census, which found the consequence class absent; the corrected run against the measured graph is reported here, and the superseded first run is retained in the artifact trail.

Three findings.

The tested configuration barely traversed. Under the pinned activation threshold of 0.0 — configuration carried from prior validated runs under the report-don't-tune rule — per-question seed sets span 377–440 of 657 engrams: roughly 60% of the corpus is activated before any edge is read, and the gold passage is already inside the seed set for 7 of 8 probes. Exactly one probe reaches gold by traversal at all (one hop, over a consequence-class edge). Arm (a) therefore measured activation-mass re-ranking over a majority-of-corpus candidate pool, not multi-hop graph traversal. This mechanically explains both the median rank of 77 — a gold passage the system already held, buried by the mass function — and the ablation's 77→74 non-effect: causal edges barely participate in a score dominated by seeding over the dense generic topology.

Linking failure and discrimination failure are both real, and now separated per probe. Three of eight probes have no non-trivial causal path to gold within six hops anywhere in the 1,195-edge causal set — their gold passages are causally disconnected from the corpus's causal subgraph, so no typed retriever, however sophisticated, had a route to follow on those probes. The remaining probes have abundant causal routes — path counts lower-bounded in the thousands to tens of thousands, a 500,000-state search cap having fired on every probe — and the arm still lost 7 of 8 overall. For this corpus and this pipeline: the causal graph was unreachable where it needed to be, and undiscriminating where it was reachable.

The type erasure is literal at the storage layer. Storage keeps one weighted slot per ordered node pair, and the typed-edge writer records the graph-level type as the generic class unconditionally — the causal class survives only as a metadata annotation. Materializing consequence edges (378 write iterations producing 646 stored records) created 413 new keys and overwrote 233 existing edges last-writer-wins, destroying their prior type and weight; the audit's one traversed path crosses exactly such a key, recorded as a 0.5-weight relation edge before the materialization and a 1.0-weight consequence edge after. §9's statement that type-blind projection erased the causal semantics is therefore not only a propagation property but a write-time one.

The audit is descriptive by pre-registration; the tier stands. What it adds is precision: the decomposition in §9 now has per-probe mechanics, and the open question this paper leaves for relation-aware retrieval carries a measured precondition — formation quality sufficient that gold is causally connected at all.

5.2 Baseline Characterization (v0.4 addendum)

A second pre-registered, descriptive follow-up — reading rules and dials frozen before any result — ran four stronger baselines on the identical frozen instrument to answer: does the §5 decoy failure survive the modern retrieval toolkit? Arms: Okapi BM25; sparse+dense hybrid via reciprocal-rank fusion (k=60); a cross-encoder reranker (BAAI/bge-reranker-v2-m3, pinned revision, CPU/fp32) over the hybrid's top-100; and, as an explicitly quarantined ceiling outside the program's deterministic-instrument class, an LLM pointwise reranker (deepseek-ai/DeepSeek-V4-Pro, temperature 0, frozen prompt, hybrid top-50, two full passes with 6/8 exact gold-rank agreement, 800 calls at the pre-set cap). Deterministic arms were double-passed bit-identical; the reference rows reproduce the published rank tables exactly.

† Decoy diagnostic was not recorded for the ablated arm in the original run.

Three observations, read under the frozen rules. The decoy failure is universal. Zero of eight questions were cleared by any of seven mechanisms; no arm — including the LLM ceiling — placed the author's own root cause at rank 1 on any question. The pre-registered strengthening condition for the question-class difficulty claim fired (ceiling median 11.5 > 5; recall@1 = 0); the pre-registered weakening condition did not. A strong relevance model demoted gold. The cross-encoder took its input's median of 26.5 down to 51.5 — reading question and passage jointly, it ranked the causal answer below where fusion had it, which is the sharpest evidence in this program that relevance semantics and causal-answer semantics diverge on this class: the same-document decoys are more answer-shaped than gold under every relevance function tested. Lexical retrieval outperformed dense, and fusion regressed recall. BM25's median beat the dense baseline (17.0 vs 28.0, with a heavy-tailed mean — rare shared technical terms either surface gold or bury it) and hybrid fusion pulled gold out of the top-20 on questions the lexical channel had caught (recall@20 0.375 vs BM25's 0.625) — a genre-specific caution against default rank fusion.

Two explanations remain live, and this instrument cannot distinguish them. Either the class is genuinely hard — isolating a causal antecedent from adjacent narrative is a synthesis task no ranking function performs — or part of the difficulty is instrument-borne: symptom-passage-as-query with author-written gold and same-document distractors may define "gold" in a way every relevance-trained system ranks behind adjacent narrative. The LLM arm's frozen prompt scored how directly a passage "states the answer," and a symptom narrative arguably does. A pointwise scorer is also not a reader — nothing here demonstrates that query-time synthesis is necessary — so the pre-registered asymmetry holds in both directions. The discriminating experiment is a larger, independently adjudicated benchmark separating natural user questions from the symptom-passage-as-query condition; it is the program's next instrument build, and the successor hypotheses of §7 are gated behind it.

6. Threats to Validity

The probe is self-built and small (n = 8); the tiers were written to survive small n (median plus win-share with ties counted against the hypothesis), and the direction of every instrument agrees, but no single number here should be quoted without its n. The corpus is one genre (SRE documents). The final experiment ran on the production extraction model (deepseek-ai/DeepSeek-V4-Pro via Nebius) while the formation lineage ran on a smaller local-class model (gemma4:31b via Ollama Cloud) — a scope condition we tracked explicitly rather than resolved. Extraction is stochastic (~70% self-reproducibility on a controlled test); the result rests on one ingest, with a confirmatory re-ingest pre-registered as the requirement had the outcome been positive — a requirement we note cuts symmetrically, and which a critic may fairly apply to the negative as well. One embedding model served as the similarity baseline in the primary comparison; §5.2 subsequently added BM25, hybrid fusion, a pinned cross-encoder, and an LLM pointwise ceiling — no relation-aware graph retriever, listwise reranker, or trained causal ranker has been evaluated, so every claim in this paper about the question class remains bounded to the mechanisms tested. The operator who adjudicated gold is the author of the thesis; blindness to condition was structural (no ranks existed at adjudication), but independent replication of the probe construction is the honest next step and the probe is published to enable it.

7. What Survived, and What We Now Believe

The substrate. Every integrity property held under adversarial self-testing: bit-identical replay across full experiment reruns, WAL durability with crash recovery, complete provenance from derived records to sources, and instrumentation that invalidated our own tainted runs before their results were read. Trustworthy memory infrastructure was the part worth keeping.

Extraction, repurposed — a hypothesis, not a measurement. Typed causal extraction produced genuine structure on real documents. Its remaining plausible value is explanation and audit — attaching why to retrieved results — a use this program designed for but never measured; we state it as the working hypothesis for the successor system, not as a finding. The distinction between an extraction pipeline's correctness and its retrieval utility was invisible to us for a year, and conflating them cost most of that year.

The invariant, weakened precisely. "No LLM at query time" was load-bearing for the recall product. §5's decoy finding removes our evidence that it is sufficient: neither ranking function we tested isolates causal answers from topical proximity. That was weaker than a falsification when first written; §5.2 subsequently evaluated a strong cross-encoder and an LLM pointwise ceiling, and the decoy failure survived both — strengthening, without proving, the reader bet (a pointwise scorer is not a reader, and §5.2's instrument caveat applies). What we now believe, stated as a design bet rather than a demonstrated necessity, is deterministic retrieval, accountable synthesis — a reader constrained to cite retrieved passages, verified mechanically.

Successor hypotheses, pre-registered, priors stated. (i) Consequence at encoding time: causal annotations appended to the embedded representation (never the immutable source), so that nearness comes to mean consequence-sharing and similarity exploits it — the successor-representation reading of the original thesis; its prior is unfavorable given §4.1, and its kill test carries the same no-reframing clause as §5. (ii) Prospective memory: typed structure applied where similarity cannot answer even in principle — deferred intentions with deterministic wake conditions, targeting an independently measured frontier-model deficit on deferred-intention tasks. Neither is claimed here; both are gated.

8. Related Work

Dense retrieval's dominance over structured alternatives at corpus scale is consistent with the broader IR literature; our contribution is not the comparison but its pre-registered, ablated, own-system form. Document expansion at indexing time (Nogueira et al., 2019) anticipates successor hypothesis (i). The cognitive framing we began from — consequence as an organizing principle of memory — has a principled home in the successor representation (Dayan, 1993; Stachenfeld et al., 2017), which locates consequence-sharing in the representation, not in query-time traversal; in retrospect the biology argued for encoding-time integration all along. Retrieval organized by expectation and outcome traces to Schank's dynamic memory (Schank, 1982). The long-conversation benchmark is LoCoMo (Maharana et al., 2024), used via a public memory-system evaluation harness. All five references were independently verified against their sources on 2026-08-30 (authors, venue, year, identifier, and cited content).

References

• Dayan, P. (1993). Improving generalization for temporal difference learning: The successor representation. Neural Computation, 5(4), 613–624. doi:10.1162/neco.1993.5.4.613
• Maharana, A., Lee, D.-H., Tulyakov, S., Bansal, M., Barbieri, F., & Fang, Y. (2024). Evaluating very long-term conversational memory of LLM agents. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 13851–13870. arXiv:2402.17753.
• Nogueira, R., Yang, W., Lin, J., & Cho, K. (2019). Document expansion by query prediction. arXiv:1904.08375.
• Schank, R. C. (1982). Dynamic Memory: A Theory of Reminding and Learning in Computers and People. Cambridge University Press.
• Stachenfeld, K. L., Botvinick, M. M., & Gershman, S. J. (2017). The hippocampus as a predictive map. Nature Neuroscience, 20(11), 1643–1653. doi:10.1038/nn.4650

9. Conclusion

We built the system our thesis demanded, gave the thesis every advantage a fair test allows, and it lost — to the boring baseline, on our own instrument, with the causal edges present, verified, and ablatable. The retrieval claim, as staked for this system, is closed. The decomposed finding is sharper than "graphs lost": known-good causal structure produced the desired retrieval behavior; automatically extracted causal structure formed but did not discriminate; and projecting typed relations into an edge-type-blind activation graph erased the distinction that made them causal. Whether a relation-aware retriever over better-formed structure can serve consequence questions remains open — this paper closes our implementation's answer, not the question. What we bought with fourteen months is a substrate we trust, a diagnosis of a genuine failure mode shared by both mechanisms we tested, two narrower hypotheses with honest priors, and a demonstration that a solo industry program can run pre-registration discipline strictly enough that its negative result is worth publishing. We think that trade was worth making exactly once, and we are publishing so that others do not have to make it again.

Data and Artifact Availability

Released with this paper: the frozen probe, the gold passages and full adjudication record including edits and drops, the pre-registration document with its ratification history and amendments, per-question rank tables and raw run results for every arm, and the corpus manifest (all thirteen source documents are public; the rendering manifest records rendering provenance and the SHA-256 list records their hashes). Path-audit addendum (§5.1): the audit results on the measured graph, its summary, the superseded first-run summary, the edge-type census across snapshots, and the census reconciliation note. Baseline-characterization addendum (§5.2): the full results and summary. Source code is not released at this time.

AI Assistance Disclosure

This research and this manuscript were produced with substantial AI assistance, disclosed here as a matter of policy. Human author (Jeff Gray): research thesis and direction; all strategic and go/no-go decisions; ratification of every pre-registered rule, threshold, and amendment before results existed; blind adjudication of all gold labels; all version-control commits; final review of, and sole responsibility for, every claim in this paper. Conversational LLM assistant (Anthropic Claude, web interface): experiment and instrument design under the author's direction; specification and prompt authoring; verification sweeps against live source code; analysis of results; drafting of findings documents and of this manuscript. Agentic coding tool (Claude Code): execution of experiments per written pre-registered instructions — harness and scoring code, ingestion runs, integrity checks, and record-keeping. Separation from the measurements: every number reported here was produced by deterministic, LLM-free instruments, double-passed for bit-identical reproduction; no language model acted as judge, answerer, or scorer for any reported result. The language models inside the system under test (ingestion-time extraction) are part of the artifact being studied and are described in §2. All figures trace to on-disk findings and pre-registration documents; the human author reviewed the manuscript against those sources and is accountable for it. An external LLM-assisted review of v0.1, solicited by the author, prompted the v0.2 scope revision; its external citations were not incorporated pending independent verification.

Changelog

• 2026-08-28 — v0.1. First full draft, authored from the program record.
• 2026-08-30 — v0.1: AI-assistance disclosure added (§3 division-of-labor paragraph; AI Assistance Disclosure section) per operator decision: disclose, always.
• 2026-08-30 — v0.1: operator decisions applied (affiliation EngramForge; models/providers named; code withheld; artifact scope fixed); June figures re-verified against the source findings doc and §4.4 corrected to distinguish the June 27-engram baseline (2.38 vs 12.38, 7/8) from the three-replicate 24-engram ceiling arm (0.875 ×3).
• 2026-08-30 — v0.2: conclusions narrowed to the staked claim following an external LLM-assisted review of v0.1. Abstract, §5, §7, and §9 rescoped to the mechanisms tested; "measured value" of extraction corrected to an unmeasured hypothesis; the record of type-aware mechanisms tested and closed expanded in §2/§4.4, with the untested query-intent-conditioned traversal named as the open residual; mechanism-bounded limitation added to §6; conclusion reframed around the decomposition (oracle structure worked; extracted structure did not discriminate; type-blind projection erased the semantics). A query→gold causal-path audit over the frozen datadir is pre-registered as an artifact addendum.
• 2026-08-30 — v0.3: path-audit addendum published as §5.1. The audit's first run executed against the pre-materialization snapshot and was caught by its own edge-type census; the corrected run on the measured graph found: seed sets of 377–440/657 under the pinned 0.0 threshold (gold already seeded in 7/8 — the arm re-ranked rather than traversed), three of eight probes causally disconnected from gold within six hops (linking failure) alongside probes with tens of thousands of causal routes that still lost (discrimination failure), and write-time type erasure with 233 last-writer-wins overwrites during consequence materialization. Five audit artifacts added to the released bundle; superseded first run retained in the trail.
• 2026-08-30 — v0.4: baseline-characterization addendum published as §5.2 (pre-registered spec; rules and dials frozen blind). BM25, hybrid RRF, a pinned cross-encoder, and a quarantined LLM pointwise ceiling all failed the decoy diagnostic 0/8; recall@1 = 0 for every mechanism; the cross-encoder demoted gold below its fusion input (26.5→51.5); BM25 outperformed dense on median while fusion regressed recall@20. Pre-registered strengthening condition for class difficulty fired; weakening condition did not. Two-explanation caveat (hard class vs. instrument-borne) stated; larger independently adjudicated benchmark named as the discriminating next instrument. §7 invariant paragraph updated to reflect that non-generative rerankers have now been evaluated. Two artifacts added to the bundle.