Sample-And-Computation-Efficient Probabilistic Model Predictive Control with Random Features

Abstract

Gaussian processes (GPs) based Reinforcement Learning (RL) methods with Model Predictive Control (MPC) have demonstrated their excellent sample efficiency. However, since the computational cost of GPs largely depends on the training sample size, learning an accurate dynamics using GPs result in slow control frequency in MPC. To alleviate this trade-off and achieve a sample-and-computation-efficient nature, we propose a novel model-based RL method with MPC. Our approach employs a linear Gaussian model with randomized features using the Fastfood as an approximated GP dynamics. Then, we derive an analytic moment matching scheme in state prediction with the model and uncertain inputs. Through experiments with simulated and real robot control tasks, the sample efficiency, as well as the computational efficiency of our model-based RL method, are demonstrated.