In pattern recognition and machine intelligence, it has been noticed that any combination of classifiers performs differently on different data sets. This has been eloquently stated by Kittler et al. (1998) “… various classifier combination schemes has been devised and it has been experimentally demonstrated that some of them consistently outperform a single best classifier. However, there is presently inadequate understanding why some combination schemes are better than others and in what circumstances.”

In this work, we introduce the first solution to this problem, which is a computational machine for selecting optimal parsimonious hybrid models of statistical classifiers of texts. This machine employs combinations of seven classifiers, including neural networks and three statistical approaches-- naïve Bayes, k-nearest neighbor, and support vector machine (with different criteria for feature selection)--to differentiate between desirable and undesirable data, including regular and attack network traffic.

The analysis concluded that optimality in parsimonious hybrid model building is data dependent and therefore a unique global optimal hybrid model does not exist. Second, our methodology selects the best model (i.e. the model with the lowest combined error) based on the data set of interest. Third, we analyzed the importance of a guidance parameter, in an error, which is a harmonic mean of precision and recall. We demonstrated that its values crucially impact the selection of the optimal hybrid model.

This work has in part been accomplished in collaboration with Dr. Khaled Alduhaiman and Sanaa Kholfi.