A Comparison of Regularization Methods for Gaussian Processes

摘要: Gaussian Processes (GPs) are classical probabilistic models to represent the results of experiments on grids points. They have numerous applications, in particular nonlinear global optimization when (typically PDE simulations) costly. GPs require inversion a covariance matrix. There many situations, optimization, density becomes higher some regions search space, which makes matrix ill-conditionned, an issue is handled general through regularization techniques. Today, need better understand and improve remains. The two most methods i) pseudoinverse (PI) ii) adding small positive constant main diagonal (which called nugget regularization). This work provides new algebraic insights into PI regularizations. It proven that averages output values variance null at redundant On opposite, lacks interpolation properties but preserves non-zero every point. However, these techniques become similar as value decreases. A distribution-wise GP then introduced interpolates distributions instead data points mitigates drawbacks regularized GPs.