Optimal Estimation of High-Dimensional Covariance Matrices with Missing and Noisy Data

Abstract

The estimation of covariance matrices is very important in many fields, such as statistics. In real applications, data are frequently influenced by high dimensions and noise. However, most relevant studies are based on complete data. This paper studies the optimal estimation of high-dimensional covariance matrices based on missing and noisy sample under the <img src="https://html.scirp.org//file/5302422-rId13.svg?20240412020625" > norm. First, the model with sub-Gaussian additive noise is presented. The generalized sample covariance is then modified to define a hard thresholding estimator <img src="https://html.scirp.org//file/5302422-rId15.svg?20240412020625" >, and the minimax upper bound is derived. After that, the minimax lower bound is derived, and it is concluded that the estimator presented in this article is rate-optimal. Finally, numerical simulation analysis is performed. The result shows that for missing samples with sub-Gaussian noise, if the true covariance matrix is sparse, the hard thresholding estimator <img src="https://html.scirp.org//file/5302422-rId17.svg?20240412020625" > outperforms the traditional estimate method.