The question this whole series exists to answer is whether language-model-like behaviour can come from something other than a large number of trained parameters. An early set of experiments turned that into a bar that a non-neural method has to clear, and the bar is a slope rather than a score: keep converting extra data into accuracy after exact-context statistics have saturated. Measured as top-1 gain per doubling of the training corpus at 4M to 8M tokens, that came out as Witten-Bell +0.005, modified Kneser-Ney +0.006, single-pass Pitman-Yor +0.008, against a small transformer's +0.021. Counting keeps improving. It improves about a quarter as fast, and that ratio is the finding. One escape hatch was left open, and it was a real one. That single-pass Pitman-Yor was explicitly a lower bound: one forward seating pass with the discount and concentration parameters pinned at 0.75 and 1.0. Its own write-up named the missing experiment, that a full hierarchical Pitman-Yor model with Gibbs sweeps and inferred hyperparameters was the strongest remaining non-neural candidate identified anywhere in the project. This is that experiment. The research repo is not public, so the figures come from its own harness and recorded runs. It does not clear the bar, and the way it fails is more useful than a pass would have been. What was actually different single-pass this seating one forward pass, committed Gibbs sweeps: un-seat and re-seat every customer given all the others table bookkeeping table counts per context and word every table's size, which is what makes a customer removable discount, concentration fixed 0.75 and 1.0 at every depth inferred per depth by auxiliary-variable sampling Everything else was held identical on purpose: the same corpus loading, the same held-out and out-of-distribution sets, the same evaluation function, order 5. The new code imports the older experiment's harness rather than reimplementing it, so the slopes compare without an asterisk. That is the only reason the headline number means anything, and it is worth more than the new model. The results Same run, same splits, 8 Gibbs sweeps. tokens model in top-1 in ppl OOD top-1 OOD ppl fit 500K hpylm 0.302 49.0 0.148 196.0 164 s 500K mkn 0.297 61.9 0.148 249.5 6 s 500K pyp-1pass 0.294 64.3 0.141 206.5 8 s 1M hpylm 0.318 40.0 0.143 200.5 324 s 1M mkn 0.310 49.8 0.138 258.1 11 s 2M hpylm 0.326 36.8 0.153 198.1 628 s 2M mkn 0.316 45.7 0.150 259.8 20 s And the slope, which is the actual question: 500K to 1M 1M to 2M hpylm +0.017 +0.008 mkn +0.013 +0.006 pyp-1pass +0.012 +0.010 The slope halves with every doubling, exactly as the weaker models' did. By 1M to 2M the full model is already at +0.008, which is the number the single-pass approximation posted at 4M to 8M, and a third of the transformer's +0.021. So the intercept moved and the slope did not. The full model holds a steady lead over modified Kneser-Ney, +0.005 then +0.009 then +0.010, and decays at the same rate. The earlier experiment had concluded that post-saturation improvement is not a smoothing problem. This extends it: it is not a seating problem either, and not a hyperparameter problem. The strongest member of the count-model family, given proper inference over table arrangements and per-depth inferred discounts, converts extra data into accuracy no faster than Witten-Bell does; the family is the limit, not the fitting. That closes the axis. A non-neural method that clears this bar will need a different kind of state, not a better-fitted n-gram hierarchy. Before this ran, the honest position was that the full model might be the answer; now nobody has to wonder, and that is the whole return on 628 seconds of fitting. Four things it did establish The single-pass approximation was leaving real accuracy behind. The full model beats it on every axis at every size: at 2M, top-1 0.326 against 0.316, perplexity 36.8 against 48.5, OOD 0.153 against 0.147. Gibbs sampling and inferred hyperparameters are worth having; they are just not worth a different slope. A single discount for every depth is a modelling error rather than a simplification. The inferred discount rises monotonically with depth, settling well above the 0.75 guess at the deep levels; the concentration falls: sweep 8 d = [0.66, 0.67, 0.79, 0.81, 0.78] theta = [1.6, 1.4, 1.4, 1.3, 1.3] This is the cheapest transferable lesson in the experiment, because it applies to any interpolated count model, including the ones this project actually ships. What the extra machinery buys is calibration, and it is remarkably stable. Perplexity comes in at 0.79x, 0.80x and 0.81x of modified Kneser-Ney's across the three sizes. Out of distribution it is 0.79x, 0.78x and 0.76x, so the advantage slightly widens there. Top-1 barely moves. A flat 20% perplexity reduction that never shows up in the ranking metric. In a mixture it wins, and the ordering follows perplexity exactly. Mixed with a transformer at a dev-tuned weight: component alone with transformer OOD alone OOD with transformer transformer only 0.351 n/a 0.156 n/a wb 0.317 0.379 0.144 0.167 mkn 0.317 0.386 0.145 0.170 hpylm 0.333 0.389 0.149 0.175 Standalone, Witten-Bell and modified Kneser-Ney tie exactly on top-1 at 0.317, so the ranking metric cannot separate them. Their perplexities differ, and so do their mixtures: 0.379 against 0.386, continuing to 0.389 with the best-calibrated model. The mixture ordering is the perplexity ordering at every step, and top-1 alone would have predicted a three-way tie. The optimal mixing weight moves too. The count side earns 0.5 instead of 0.6 once it is better calibrated, which is the mechanism behind the rule rather than a restatement of it: a mixture trusts a component in proportion to how honest its probabilities are. The best number in the project, and what it cost Swapping this model into the three-way recipe, with the corpus half as the only change: WB HPYLM delta corpus alone 0.316 0.330 +0.014 with transformer 0.373 0.387 +0.014 three-way 0.376 0.386 +0.010 corpus alone, OOD 0.145 0.152 +0.007 three-way, OOD 0.182 0.192 +0.010 The Witten-Bell arm reproduces its published 0.376 and 0.182 exactly, which is what makes the delta readable: the harness is unchanged and the comparison is same-run rather than against a figure from a different session. 0.192 out of distribution is the best result anywhere in this project. There is a subtler finding underneath it. What the document cache adds on top of corpus plus transformer, with the two corpus models: in-distribution OOD with WB +0.003 +0.009 with HPYLM −0.001 +0.011 An earlier experiment found the cache and the transformer partially redundant in-distribution and complementary out of it. With a well-calibrated corpus model the in-distribution contribution vanishes entirely, a hair below zero, while the out-of-distribution contribution is undiminished and slightly larger. Document statistics are information the corpus cannot contain, so no amount of better corpus modelling substitutes for them; in-distribution, though, part of what the cache was supplying was just better-calibrated local probability, and a better corpus model already has that. Do not ship it Half a point in the two-way mixture, a point in the three-way. That is bought with roughly 30x the fitting time and about 1.4 GB of resident memory at 2M tokens, for a model that must be refitted from scratch whenever the corpus changes. Modified Kneser-Ney gets within half a point at a thirtieth of the cost, which is why the code-completion tool in this project keeps using a cheap model and should. The value here is mechanistic rather than an upgrade: calibration decides mixtures, now measured across four count models, and better calibration substitutes for the document cache in-distribution but not out of it. The limits, which matter for the headline Eight sweeps is not convergence. The hyperparameters were still drifting at sweep 8, so these numbers are a floor for the full model rather than its ceiling. That does not rescue the slope conclusion, and the reason is worth stating: a floor that decays at the same rate as its baseline is still decaying at the same rate. The ladder stops at 2M because memory does, at 3.0 GB resident, and the original bar was quoted at 4M to 8M. The 1M to 2M slope here already matches what the weaker models showed at 4M to 8M, which is why the conclusion is phrased as closing the axis rather than as a measurement at the same scale. It is also one domain, order 5, English. The base distribution is uniform over the vocabulary; a unigram or character-level base would probably help out of distribution, and is untested.