General
Sample-Efficient Model Predictive Control via Proximal Policy Optimization
arXiv cs.SY0 views
arXiv:2611.23544v1 Announce Type: new Abstract: Sampling-based model predictive control (MPC) enables real-time control of complex nonlinear systems but often incurs substantial computational costs due to repeated trajectory simulations. In this paper, we bridge policy optimization in reinforcement learning and sampling-based MPC to improve computational efficiency through sample reuse. Starting from a KL-regularized distributional formulation of trajectory optimization, we derive an exact performance-difference identity based on a signed exponential advantage function. This identity motivates two algorithms, PPO-MPPI and SPO-MPPI, which adapt proximal policy optimization techniques to sampling-based MPC, enabling multiple optimization updates using previously evaluated trajectories without additional forward simulations. We establish population-level monotonic improvement guarantees for both methods and prove convergence to first-order stationarity for PPO-MPPI under fixed-covariance Gaussian sampling distributions. Experiments on quadruped velocity tracking and quadrotor path tracking demonstrate that the proposed methods achieve competitive control performance at substantially lower computational cost than multi-stage sampling approaches. Our results establish a principled connection between policy optimization and sampling-based MPC and highlight the potential of trajectory reuse for sample-efficient predictive control.
Source:https://arxiv.org/abs/2611.23544PreviousReinforcement Learning Optimized Flight Control for Interception of High-Altitude Maneuvering TargetsNextADSPICE: A Method of Automatic Differentiation for Gradient-Based Lumped Circuit Optimization with Native SPICE
Related posts
Generative modeling of conditional probability distributions on the level-sets of collective variables
0arXiv stat.ML
Tired of Endless Subscriptions? Here’s How to Stream Your Favorite Movies for Free in HD
0dev.to WebDev
Show HN: Port.fm – Internet radio tuned by decade, energy, or location
0Hacker News Show
Show HN: Snowchive – Snowden archive of all published documents
0Hacker News Show
Show HN: A small coding tool for kids
0Hacker News Show
Fanatics Promo Code: 10% Off
0Wired