Bias of the Random Forest Out-of-Bag (OOB) Error for Certain Input Parameters

Abstract

Random Forest is an excellent classification tool, especially in the –<i>omics</i> sciences such as metabolomics, where the number of variables is much greater than the number of subjects, <i>i.e</i>., “<i>n</i> << <i>p</i>.” However, the choices for the arguments for the random forest implementation are very important. Simulation studies are performed to compare the effect of the input parameters on the predictive ability of the random forest. The number of variables sampled, <i>m-try</i>, has the largest impact on the true prediction error. It is often claimed that the out-of-bag error (OOB) is an unbiased estimate of the true prediction error. However, for the case where <i>n</i> << <i>p</i>, with the default arguments, the out-of-bag (OOB) error overestimates the true error, <i>i.e</i>., the random forest actually performs better than indicated by the OOB error. This bias is greatly reduced by subsampling without replacement and choosing the same number of observations from each group. However, even after these adjustments, there is a low amount of bias. The remaining bias occurs because when there are trees with equal predictive ability, the one that performs better on the in-bag samples will perform worse on the out-of-bag samples. Cross-validation can be performed to reduce the remaining bias.