You fine-tuned at full precision. You serve in FP8. Those were two different models.

transformers v5.22.0 made FP8 models trainable with LoRA. The interesting part is the workflow it removes, not the feature it adds.

You fine-tuned at full precision. You serve in FP8. Those were two different models.

The line

From the transformers v5.22.0 release notes, 29 October 2026:

FP8 quantized models can now be used for adapter training (such as LoRA with PEFT)

It landed with FP8 backward support — gradients, not just the forward pass. The notes add that "Backward also dequantizes experts in chunks and skips autograd at inference."

Quantised models have mostly been inference artefacts. A thing you produce at the end, to make serving affordable. This makes the cheap-to-serve version also the one you can adapt.

The workflow it fixes

If you fine-tune on hardware you can actually afford, you have probably run this loop:

1. Take a base model at full precision — or as close as you can manage.
2. Train a LoRA adapter against it.
3. Evaluate. The numbers look good. You write them down.
4. Quantise for serving, because full precision does not fit the GPU you have.
5. Ship.

Step 4 changes the weights the adapter was optimised against. The artefact you measured in step 3 is not the artefact your users reach in step 5.

Everyone knows this. Almost nobody adjusts for it, because the alternative — evaluating the quantised build properly — costs another pass and the gap is usually small.

Why "usually small" is the problem

A large regression is easy. It shows up immediately, someone bisects it, you find it.

A small one is corrosive:

• Your eval report describes the wrong artefact. Any number you quote about the fine-tune is a number about a model you did not deploy.
• Regressions get misattributed. Quality drops after a serving change, so the serving stack gets blamed. The adapter is never re-examined because "the eval was fine."
• Two variables moved at once. The adapter changed and the precision changed between measurement and production. You cannot attribute an effect to either.
• It compounds across iterations. Each cycle re-measures at full precision and re-ships quantised, so the gap is never once measured directly — only inherited.

Training the adapter against the same quantised base you serve collapses that into one variable. That is the real value here, and it is a methodology win rather than a performance one.

What I would hold to regardless of this release

Evaluate the artefact you ship. If your eval ran against a different precision than production, that belongs in the eval report as a caveat, not in a footnote nobody reads.

Record the quantisation format next to the adapter. An adapter alone is an incomplete artefact. adapter_v3/ tells you nothing; adapter_v3_fp8_base/ tells you what it was fitted to.

If you must train at one precision and serve at another, measure the gap on purpose. Run the eval set against both builds, once, and write down the delta. Make it a release gate. A known 2% gap is a managed risk; an unknown one is a future incident with no obvious cause.

Move one variable at a time. Old engineering advice, routinely ignored in ML because runs are expensive. The irony is that the debugging caused by moving two at once costs more than the extra run.

Two hard rules

The model you evaluated is the only model you can make claims about. Not the one you meant to evaluate, not the one that is nearly identical. The one the numbers came from.

An adapter is not portable across precisions by assumption. It is portable if you measured it.

The payoff

Attribution. When output quality moves, you can say which change moved it — and that is the difference between an engineering process and a sequence of hopeful experiments.

Honest limitation

I am reading release notes and reasoning from them. I have not trained an FP8 adapter and I am not reporting benchmark results.

I also want to be precise about what this does not claim: training against a quantised base narrows the train/serve gap, it does not eliminate it. Optimiser behaviour, kernel implementations and hardware still differ between your training box and your serving box. "Same precision" is not "same artefact." It is one fewer difference, which is worth having and is not a guarantee.

Checkable references

• transformers v5.22.0 (29 October 2026) — "FP8 quantized models can now be used for adapter training (such as LoRA with PEFT)"; FP8 backward support
• transformers v5.21.0 (26 October 2026) — FP8 backend for the quantized KV cache

One open question

When you report a fine-tune's results, are those numbers from the quantised artefact you actually serve — and if not, do you know the size of the gap?