Abstract: Properties of convex correcting procedures (CCP) on ensembles of predictors are studied. CCP calculates integral solution as convex linear combination of predictors’ prognoses. Structure of forecasting squared error and generalized error are analyzed. At that generalized error is defined as mean of squared error at Cartezian product of forecasted objects space and space of training sets. It is shown that forecasting squared error, bias and variance component of generalized error have similar structure. Search of optimal CCP coefficients is reduced to quadratic programming task which is solved in terms of ensemble superfluity. Ensemble is considered superfluous if some members can be removed without loss of forecasting ability. Necessary and sufficient conditions of superfluity absence are proven. A regression method based on the described principles has been developed. Its concepts as well as testing results are shown revealing CCP’s significant superiority over stepwise regression.

Keywords: forecasting, bias-variance decomposition, convex combinations, variables selection

ACM Classification Keywords: G.3 Probability and Statistics - Correlation and regression analysis, Statistical computing

Link:

OPTIMAL FORECASTING BASED ON CONVEXCORRECTING PROCEDURES

Oleg Senko, Alexander Dokukin

http://foibg.com/ibs_isc/ibs-16/ibs-16-p08.pdf