AI & ML
TTHE: Test-Time Harness Evolution
arXiv cs.SE0 views
arXiv:2607.08124v3 Announce Type: replace Abstract: The behavior of an LLM agent is determined not only by the underlying model, but also by its harness: the executable program that constructs context, invokes tools, verifies intermediate results, and recovers from failures. Existing approaches typically optimize harnesses on training or development data and freeze the resulting programs at test time, limiting adaptation to shifts in task distributions, failure modes, and tool interactions. We ask whether the harness can instead be optimized during evaluation using unlabeled execution traces. We introduce Test-Time Harness Evolution (TTHE), which treats the executable harness as the state of test-time adaptation. On each unlabeled test batch, TTHE iteratively evolves a population of candidate harnesses through agentic proposers that reason over execution traces and explore diverse edit directions. A judge selects a candidate using execution traces and execution-derived proxy signals. The selected harness determines the current batch's outputs and initializes adaptation on the next. Crucially, the solver, proposers, and judge operate through different harnesses around the same frozen LLM, without weight updates, access to final-answer labels, or a separately trained adaptation model. Across six execution-grounded benchmarks spanning text-to-SQL, competitive programming, software engineering, data-science coding, terminal-based tasks, and agentic tool use, TTHE improves fixed ReAct-style baseline harnesses through persistent, inspectable program adaptation. These results recast test-time adaptation for LLM agents as evolution over executable control programs and identify execution-derived proxy reliability as a central challenge for robust unsupervised agent improvement.
Source:https://arxiv.org/abs/2607.08124Related posts
Test DOCX Breaks as a Many-to-One Conversion
0dev.to JavaScript
A Bayesian Optimized Biquad Filter-Burg Model for Real-Time EEG Phase Estimation
0arXiv eess.SP
Thought-deciphering via hierarchical EEG-to-text alignment
0arXiv eess.SP
Neuromorphic Decoding and Dynamical Image Reconstruction from Populations of Spiking Retinal Ganglion Cells
0arXiv eess.SP
Benchmarking Sensor-Fault Robustness in Forecasting
0arXiv eess.SP
Consistent Geometric Deep Learning via Hilbert Bundles and Cellular Sheaves
0arXiv eess.SP