Asymptotic Properties of Hyperparameter Estimators by Using Cross-Validations for Regularized System Identification

Biqiang Mu1, Tianshi Chen2, Lennart Ljung3

  • 1Chinese Academy of Sciences
  • 2The Chinese University of Hong Kong, Shenzhen, China
  • 3Linkoping Univ.

Details

11:00 - 11:20 | Mon 17 Dec | Splash 10 | MoA18.4

Session: Identification I

Abstract

This paper studies the asymptotic properties of the hyperparameter estimators including the leave-$k$-out cross validation (LKOCV) and $r$-fold cross validation (RFCV), and discloses their relation with the Stein's unbiased risk estimators (SURE) as well as the mean squared error (MSE). It is shown that as the number of data goes to infinity, both the LKOCV and RFCV share the same asymptotic best hyperparameterminimizing the MSE estimator as the SURE does if the input is bounded and the ratio between the training data and the whole data tends to zero. We illustrate the efficacy of the theoretical result by Monte Carlo simulations.