AI & ML
Option-GRPO: Hierarchical RL Against AI Slop in Code Generation for Unseen SDKs
arXiv cs.SE0 views
arXiv:2611.23633v1 Announce Type: new Abstract: Large language models increasingly produce AI slop in generated code: verbose, redundant, non-idiomatic implementations that pass functional checks but bloat codebases. The problem compounds on SDKs released after the model's training cutoff, where the model has neither idiomatic priors nor compositional understanding. We present Option-GRPO, a hierarchical reinforcement learning framework that adapts a code LLM to an unseen SDK from its source code alone, without human-written usage examples or test cases, and that explicitly optimizes for both functional correctness and code elegance. Our key insight is that SDK source code contains compositional structure that defines a hierarchy of reusable code patterns (code options) which simultaneously serve as the building blocks of a curriculum and the action space for efficient policy optimization. We show that code options induce a valid semi-Markov decision process, give conditional bounds under which shorter option description length implies more concise and, in expectation, more API-efficient programs, and analyze under stated assumptions how the option hierarchy improves credit assignment by a factor of $\sqrt{T/K}$. Using Group Relative Policy Optimization (GRPO) with execution-based rewards at each curriculum level, Option-GRPO discovers SDK idioms, composes them into patterns, and assembles patterns into complete programs. Experiments on 12 unseen SDKs with a 32B-parameter base model demonstrate 61.4% pass@1, outperforming flat GRPO by 8.7 points, ExploraCoder by 11.2 points, and ReCode by 6.3 points. Option-GRPO produces solutions that use 34.7% fewer API calls and are 27% shorter than flat-GRPO outputs while maintaining higher correctness, directly attacking the slop problem.
Source:https://arxiv.org/abs/2611.23633PreviousRecord Sufficiency: What a Decision Record Must Preserve When an Unsettled Claim Becomes an Operative StatusNextSafe Personal RSI: Evaluating and Repairing Personal-Agent Updates
Related posts
Thống kê hiệu suất Al Ahli vs Al Nassr: Phép thử cực đại cho hàng công chủ nhà
0dev.to Data
Phân tích số liệu vòng 10 Bundesliga: Thách thức chiến thuật của FC Koln trước Bayern Munich
0dev.to Data
Dấu ấn chiến thuật của Hansi Flick: Biến số Frenkie de Jong giúp Barcelona ngược dòng trước Atletico Madrid
0dev.to Data
PMO Support Services in the UAE: What Effective Program Governance Actually Requires
0dev.to News
The draft is not the send
0dev.to Discuss

My First GitHub Issue & My First Forem (DEV.to) Issue 🚀
0dev.to Discuss