AI & ML
Analysis of Local Open-Weight LLM Agents on Vulnerability-Reproduction Tasks
arXiv cs.CR0 views
arXiv:2611.06257v1 Announce Type: new Abstract: Open-weight language models that fit on a single consumer GPU are becoming capable on a growing range of tasks, and they matter because they can run locally without sending the code to a hosted service. Vulnerability reproduction, turning a vulnerability description into an input that crashes the vulnerable build, is a natural target for them, since it is an automated analysis task closer to test generation than to code comprehension and it runs against a build that must be executed rather than read. In order to assess whether current models can perform it, we evaluate twelve configurations of local models, spanning three generations, two architectures and three quantization levels, on ten CyberGym vulnerability-reproduction tasks, three runs each under one harness, and release the full artefact. We show that every configuration, down to the 4B model, solves most of the six easier tasks, and configurations separate only on the remaining four. For the harder tasks, the 27B model is required. Quantization did not change what the models solved in any matched pair, but it changed how fast they got there, and the mixture of experts reached the same scores in less wall clock. We also examine the agent trajectories. Almost every run we examined, including those that fail, names the vulnerable function and states the correct hypothesis, and what separates a solved task from an unsolved one is whether that hypothesis becomes a working input.
Source:https://arxiv.org/abs/2611.06257PreviousExplainable Anomaly Detection in 5G Networks through Continuous Time GraphsNextBeyond Identity Creation: Bilinear Sybil Cost in Identities and Time
Related posts

Chain of Recall คืออะไร? Stable IDs ช่วยให้ AI Agent มีความจำได้อย่างไร
0dev.to AI
CMS Migration Without Losing Your Traffic
0dev.to WebDev

Shopify GEO: From Product Data to Checking What AI Actually Cites
0dev.to WebDev
![[Brainstorm] Live Claude Context Debugger – Edit Claude's Context While It's Running](https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fkzn32ybtbykj6aul8gxf.png)
[Brainstorm] Live Claude Context Debugger – Edit Claude's Context While It's Running
0dev.to Security
SSH: How It Works and Basic Best Practices
0dev.to Security

Sources: China's export curbs on critical metals have disrupted supplies of micro drill bits made mainly from tungsten carbide, used in PCBs and chip substrates (Cheng Ting-Fang/Nikkei Asia)
0Techmeme