Title:Kalman Filter-based Heuristic Ensemble: A New Perspective on Ensemble Classification Using Kalman Filters

Abstract:A classifier ensemble is a combination of multiple diverse classifier models whose outputs are aggregated into a single prediction. Ensembles have been repeatedly shown to perform better than single classifier models, therefore ensembles has been always a subject of research. The objective of this paper is to introduce a new perspective on ensemble classification by considering the training of the ensemble as a state estimation problem. The state is estimated using noisy measurements, and these measurements are then combined using a Kalman filter, within which heuristics are used. An implementation of this perspective, Kalman Filter based Heuristic Ensemble (KFHE), is also presented in this paper. Experiments performed on several datasets, indicate the effectiveness and the potential of KFHE when compared with boosting and bagging. Moreover, KFHE was found to perform comparatively better than bagging and boosting in the case of datasets with noisy class label assignments.