Markov blanket-based universal feature selection for classification and regression of mixed-type data

Abstract

Feature selection has been successfully applied to improve the quality of data analysis in various expert and intelligent systems. However, because most real-world data nowadays come with mixed features, traditional feature selection approaches that are mainly designed to handle single-type data are not suitable for this situation. In addition, most of existing methods are only applicable to a specific problem, either classification or regression. Therefore, it is an urgent need to develop a universal feature selection method that can be applied to classification and regression with mixed-type data. In response to this, our paper presents a new feature selection method based on a Markov blanket (MB) called Mixed-MB. The key idea behind this is to embed a likelihood ratio-based generalized conditional independence test into an efficient MB search algorithm to find the minimal set of features to fully explain the target variable on mixed-type data. This new MB feature selection method eliminates the weakness of existing MB feature selection method that it only can handle single-type data, while maintaining its strengths such as theoretical soundness, simplicity, speed, and versatility. Experimental results on real-world data sets with mixed features demonstrate that the proposed method is effective for improving the accuracy of prediction models in both classification and regression. It is also shown to be able to yield more accurate results with fewer features than other methods. We believe that Mixed-MB will be widely used in expert and intelligent systems that utilize various data to create value since it can be applied to any type of data and problem.