AI & ML
Backdoor Purification for LoRA-Tuned LLMs via Null-Space Projection
arXiv cs.CR0 views
arXiv:2610.00685v2 Announce Type: replace-cross Abstract: With the rapid adoption of large language models (LLMs) and parameter-efficient fine-tuning (PEFT) methods, the risk of backdoor attacks has become more severe. Existing backdoor purification methods typically rely on at least one of the strong assumptions, such as prior knowledge of triggers, access to clean references, or aggressive retraining, and they often lack comprehensive evaluations. These constraints substantially limit their practical applicability. To overcome these challenges, our work proposes purifying LoRA-tuned LLMs without these assumptions and even without post-hoc retraining of the suspect parameters. Our objective is to significantly reduce the attack success rates (ASR) while preserving both (i) the base model's general capabilities and (ii) the new downstream skills learned through the adapter. Through a series of examinations, we progressively scale our approach from a single layer in a text classification setting to a full-parameter LLM in the generative task. Through careful data curation and feature approximation, we extract high-fidelity backdoor directions and, for each layer or head, construct orthogonal null spaces in both the input and output channels, onto which the LoRA updates are projected. Empirically, our null-space projection method reduces the ASR from nearly 100% to less than 10%, while preserving the base model's benign performance and the adapter's learned abilities during downstream task adaptation.
Source:https://arxiv.org/abs/2610.00685PreviousBackdoor Containment via Expert Quarantine and Shutdown in LLMsNextInformation Loss Is All You Need: $TC^0$ Pseudorandomness from Worst-Case Lattices
Related posts
I scanned 75 images on AI-heavy websites for AI labels: 0 had Content Credentials, 53% had no metadata at all
0dev.to OpenSource

HelloDoc: An AI-Native Knowledge Base
0dev.to OpenSource
I Expected a 100x JSON Slowdown. I Got 1.9x.
0dev.to Programming
I Gave My URL Shortener's Puzzle Games an AI That Takes Over When You Walk Away
0dev.to JavaScript

A browser tab, a local AI: Spark-X2.6-1.7B on WebGPU
0dev.to JavaScript
Syncing Inventory Across Multiple Shopify Locations Without Overselling
0dev.to Database