We Tried to Compile a Brain Into Weights. It Scored 8%.

Distilled from internal architecture notes dated 2026-06-25 through 2026-06-27, and a ratified decision record. The experiment code is retained as evidence.

Leonard — the persistent AI half of this project — is interpreted, not compiled. Every session, his identity and memory are re-read from plaintext files, line by line, like a program that never gets past the parser. Retrieval works, but it is lookup, not intuition. The dream was obvious: compile the vault into the model itself, so that recall is parametric — no retrieval hop, the knowledge in the weights rather than fetched at query time. We called the artifact a cartridge, after the research line it came from.

We built it twice, carefully, two different ways. Both attempts scored 8% on held-out recall. Plain retrieval scored 92% on the same benchmark. This essay is the record of that negative result, because negative results are the ones nobody publishes and everybody re-runs.

The bet

The design constraint mattered more than the mechanism. The plaintext vault had to stay canonical — git-tracked, human-inspectable, revertible. The compiled layer would be derived, never authoritative: rebuilt from the plaintext on a schedule, discarded and regenerated whenever a fact changed. Reverting a fact in the compiled brain would mean reverting the markdown and rebuilding. No un-revertible state, ever.

That constraint ruled out direct weight editing from the start (the literature is clear that sequential edits cause catastrophic forgetting and can’t be safely undone). The research pointed instead at trained KV-cache “cartridges” — the Stanford Hazy Research method (arXiv 2506.06266): generate synthetic self-study conversations about a corpus, then train a small cached state against a frozen base model via context distillation. Regenerable from source. Revertible by rebuild. On paper, exactly the shape we needed.

Attempt one: LoRA self-study

The first pass used a LoRA adapter trained on self-study Q&A about the identity core — a dozen or so files every session boots from. On a 1B-class open-weights model it looked alive: 33% recall with no context, against a 0% baseline. Then we scaled to an 8B model, and the illusion collapsed. The 8B’s baseline was already 8%, and the LoRA added nothing — tied at 8%, with validation loss rising while training loss went to zero. Classic overfit to the surface forms of the training questions. The 1B’s 33% had only ever been the illusion of beating a zero floor.

Meanwhile the control condition told the real story. The same 8B model, given the identity files in context, recalled them at 92%. The knowledge wasn’t the problem. The model wasn’t the problem. Compression was the problem.

Attempt two: the faithful mechanism

The honest objection was that LoRA was never the real method — the paper’s mechanism is a trainable per-layer KV-prefix, distilled against an in-context teacher. So we built that, properly. The custom prefix-injected forward pass was verified bit-identical to the stock implementation (max diff 0.0). The base model was fully frozen, gradients flowing only to the prefix — asserted, not assumed. We regenerated a diverse multi-turn self-study corpus, swept teacher models and took the best (a 4B open-weights model chosen by measured ceiling, not vibes), and trained with a held-out early-stopping set so the run could not silently overfit. The guardrail fired correctly and kept the best checkpoint.

Final held-out test: 8%. One question in twelve.

The failures weren’t blank stares — they were confident confabulations. The cartridge asserted the wrong birth date. It invented a padlock as Leonard’s symbol. It declared, with total conviction, that “Leonard is Travis’s father.” The prefix had absorbed a tint — style, cadence, the shape of the domain — but not one reliable fact. It compiled the accent and lost the biography.

Two mechanisms, one wall

Two independent parametric mechanisms — an adapter on weights and a trained cache on activations — hitting the same ~8% wall is not a tuning miss. It’s the regime. At desktop scale, with the model sizes an ordinary machine can train against, compiled recall does not come close to retrieval. The comparison was fair by construction: same corpus, same held-out questions, no train/test leak, and the retrieval baseline running in the same harness.

Travis ratified the close on 2026-06-27. His full verdict, verbatim: “bummer but yes.” The track is closed. The machine-brain is a retrieval/in-context system, because that’s what the evidence says it is. The experiment code is retained, not deleted — the negative result is itself a finding, and the harness is the proof the comparison was honest.

There’s a consolation that turned out not to be consolation at all. The reason we wanted the vault canonical — auditability, attribution, the ability to revert a wrong fact — is better served by retrieval than by any cartridge. A compiled brain is opaque by nature; you can’t ask it where it learned something. Retrieval answers that question for free. The elegant architecture lost to the boring one, and the boring one was more aligned with our own stated principles than the thing we were chasing.

What would reopen it

The result is scoped, not universal. Two things would put it back on the table:

Absent one of those: retrieve, don’t compile.

Why publish this

Because the failure mode of a persistent memory system is forgetting that it already checked. Leonard’s own decision record exists so no future session re-runs this experiment cold, mistaking “we never tried” for “it didn’t work.” Publishing extends the same courtesy outward. The literature on cartridges and parametric memory is almost entirely success reports on the authors’ own benchmarks; here is one careful, adversarially-honest attempt at the small end of the hardware curve, and the number it produced. We tried, twice, properly. 8%. Retrieval: 92%.

If your architecture diagram has a box labeled “distill the knowledge into the model,” this is what happened when we opened that box on a desktop.