Efficient Gaussian Process Based on BFGS Updating and Logdet Approximation

Gaussian process (GP) is a Bayesian nonparametric regression model showing good performance in various applications. However, its hyperparameter-estimation procedure suffers from numerouscovariance-matrix inversions of prohibitively $O(N^3)$ operations. In this paper, we propose usingthe quasi-Newton BFGS $O(N^2)$-operation formula to update recursively the inverse of covariance matrix at every iteration. As for the involved $\log\det$ computation, a power-series expansion based approximation and compensation scheme is proposed with only $50N^2$ operations. A number of numerical tests are performed based on the 2D-sinusoidal regression example and the Wiener-Hammerstein identification example. It is shown that by using the proposed implementation, more than $80\%$ $O(N^3)$ operations are eliminated, and the speedup of $5\sim9$ can be achieved.