AI & ML
Prior Knowledge or Search? A Study of LLM Agents in Hardware-Aware Code Optimization
arXiv cs.SE0 views
arXiv:2605.19782v2 Announce Type: replace-cross Abstract: LLM discovery and optimization systems are increasingly applied across domains, implementing a common propose-evaluate-revise loop. Such optimization or discovery progresses via context conditioning on received feedback from an environment. However, as modern LLM agents are increasingly complex in their structure, it is difficult to evaluate which components contribute the most, and when and how this exploration may fail. We answer these questions through three controlled experiments. Our findings: (1) In pure black-box optimization, LLMs act as greedy optimizers. (2) In zero-shot kernel generation, providing explicit input-size information has no measurable effect, models converge to the same kernel parameters regardless of size or temperature, as though the size instruction were invisible. Moreover, when tasked to perform kernel optimization for uncommon kernel sizes, performance sharply degrades regardless of the language used. (3) In feedback-loop kernel optimization, CUDA improves monotonically under iterative feedback, while TVM IR actively degrades, which demonstrates that kernel optimization degrades when models operate with low-density language. Our results conclude that LLMs in code optimization tasks highly depend on pretrained priors rather than provided feedback or agentic structure.
Source:https://arxiv.org/abs/2605.19782PreviousLive Noise Fingerprints for Quantum Program AnalysisNextWhat Does a Harness Buy? Tokens, Mostly
Related posts
How do you perform AI code review?
0dev.to Git
gpt-5 and o3 shut down December 11: a find-and-replace would rewrite 274 table rows
0dev.to TypeScript
Exporting App Store reviews from every country in Node.js: 6 things the RSS feed doesn't tell you
0dev.to JavaScript
What Does a Harness Buy? Tokens, Mostly
0arXiv cs.SE
Live Noise Fingerprints for Quantum Program Analysis
0arXiv cs.SE
Biomedical Knowledge Composition: A Software Engineering Perspective
0arXiv cs.SE