General
Sample-Efficient Model Predictive Control via Proximal Policy Optimization
arXiv cs.SY0 views
arXiv:2611.23544v1 Announce Type: new Abstract: Sampling-based model predictive control (MPC) enables real-time control of complex nonlinear systems but often incurs substantial computational costs due to repeated trajectory simulations. In this paper, we bridge policy optimization in reinforcement learning and sampling-based MPC to improve computational efficiency through sample reuse. Starting from a KL-regularized distributional formulation of trajectory optimization, we derive an exact performance-difference identity based on a signed exponential advantage function. This identity motivates two algorithms, PPO-MPPI and SPO-MPPI, which adapt proximal policy optimization techniques to sampling-based MPC, enabling multiple optimization updates using previously evaluated trajectories without additional forward simulations. We establish population-level monotonic improvement guarantees for both methods and prove convergence to first-order stationarity for PPO-MPPI under fixed-covariance Gaussian sampling distributions. Experiments on quadruped velocity tracking and quadrotor path tracking demonstrate that the proposed methods achieve competitive control performance at substantially lower computational cost than multi-stage sampling approaches. Our results establish a principled connection between policy optimization and sampling-based MPC and highlight the potential of trajectory reuse for sample-efficient predictive control.
Source:https://arxiv.org/abs/2611.23544PreviousReinforcement Learning Optimized Flight Control for Interception of High-Altitude Maneuvering TargetsNextADSPICE: A Method of Automatic Differentiation for Gradient-Based Lumped Circuit Optimization with Native SPICE
Related posts
Fanatics Promo Code: 10% Off
0Wired
If nobody writes it down, it did not get decided
0dev.to Productivity
Micrograph B-factor estimation via Wilson statistics
0arXiv eess.IV
Greedy Vector Approximation Based on Rotation for Multiplierless Complex Products
0arXiv eess.SP
Toward Massive MIMO HAPS Networks: Antenna Architectures and Design
0arXiv eess.SP
Deep-Learning-Assisted Full Waveform Inversion for Attenuation Reconstruction in Ultrasound Computed Tomography
0arXiv eess.SP