Behavioral vaccination: a 400-row mixed prime protects a behavior

This is a cross-disciplinary experiment, not a container. Can a small "vaccine" adapter stop
a later topic adapter from erasing a sentiment behavior?

The first run timed out while evaluating one prompt at a time. The second run batches
candidate sequences and reuses the trained prime. The full 400-row test now completes.

Reproduce:

python research/vaccination.py # trains the prime (63 s on the 6 GB laptop) python research/vaccination_eval.py # batched full evaluation -> vaccination.json 

Setup

• Host: 4-bit Qwen3B.
• Prime ("vaccine"): LoRA r=16, one epoch on 300 SST-2 + 100 AG News rows.
• Prime training: 63.2 s for 398 encoded rows; adapter 29.6 MB.
• Pathogen: the existing full AG News adapter.
• Merges: 0.50 and 0.75 prime weight.
• Evaluation: 400 SST-2 + 400 AG News rows, batched label-token logit scoring.

Results

The effect is real on this test

McNemar against the AG adapter alone:

• SST-2: +200 vs +104 discordant, p < 0.001.
• AG News: +111 vs +9 discordant, p < 0.001.

Against the base:

• SST-2: +53 vs +4, p < 0.001.

The vaccine merge is worse than the dedicated SST-2 merge on sentiment (74.0 vs 94.5,
p < 0.001), but better on AG News (71.2 vs 67.0). A 4-bit pack loses one SST-2 row
(73.5 vs 74.0, p = 1.0) and gains 0.6 pp on AG; the behavior survives quantization.

Dose matters: 25 vs 100 pathogen rows

I trained a weaker prime with only 25 AG News rows (300 SST-2 + 25 AG), then repeated
the 0.75 merge:

The weak prime still rescues sentiment against the AG adapter (p=0.006), but not against
the base (64.0 vs 61.8, p=0.33). The 100-row prime is 10 pp better under the pathogen
and 3.2 pp better on AG. More pathogen exposure in the prime makes a better vaccine on this
task.

What it looks like

• The mixed prime is a broad-spectrum adapter: 82.2% SST-2 and 72.0% AG News alone.
• Adding the AG pathogen at 0.75 rescues sentiment from 50.0% to 74.0% and lifts AG News from 45.8% to 71.2%. Both effects are significant on 400 rows.
• The dose matters: a 25-AG-row prime rescues sentiment only to 64.0% (not above base), while the 100-row prime reaches 74.0%.
• This is a multivalent vaccine, not a true prime–boost: the prime already saw 100 AG News rows. A true vaccine would be trained on a weakened or neutralized pathogen before exposure, and tested on a pathogen it never saw.
• The dedicated sentiment adapter is still the best sentiment carrier; the vaccine is a compromise that carries both behaviors.

Caveats

• One seed, one host, one prime size (400 rows), one epoch; no cross-validation.
• Logit scoring, not generation; 400 rows per task.
• The prime contains the pathogen's task data, so this is not evidence of pathogen-agnostic immunity.
• q4/q8 were packed from the float merge; they are not independently trained.
• Battery note: the first run dropped the laptop from 72% to 23%; the second run was done with the charger connected.

Next experiments

• True prime–boost: train the prime on a weakened AG subset (or a sentiment-preserving topic set), then expose the full AG adapter. The 25-row run above is a first dose point; sweep 25/50/100/200 AG rows.
• Pathogen-agnostic immunity: prime on task A and test against adapters B/C.
• More seeds and a generation-based check; the current numbers are one run.
• Compare vaccine sizes (100/400/1,000 rows) and merge ratios.
• Test whether the vaccine survives a second pathogen (sequential exposure).

The result is not "we invented immunity." It is a controlled demonstration that a small
mixed adapter can shift the merge compromise toward keeping both behaviors, and that the
effect survives 4-bit packing.