Fuzzy-Rough Approaches for Pattern Classification: Hybrid measures, Mathematical analysis, Feature selection algorithms, Decision tree algorithms, Neural learning, and Applications

The primary objective of any supervised learning technique is to learn an unknown function (or at least a good approximation of it) from a set of observed input-output patterns. Pattern classification is a special case of function approximation, where each pattern is assigned to a particular class, i.e., the output in classification problem is one of the discrete values corresponding to class rather than real-valued function in regression. Precisely, in a classification problem output belongs to one of the discrete classes, while in a regression problem output belongs to a set of real number R. This book discuss various fuzzy-rough approach to pattern classification, and develops some hybrid measures and algorithms for attribute (or feature) selection and induction of fuzzy decision trees. Hybrid fuzzy-rough algorithms derive best benefits of both the worlds; fuzzy systems and rough sets theory. Fuzzy systems are well known for their ability to handle vagueness in the data. On the other hand, rough sets are pure data driven knowledge discovery tools capable of handling ambiguity very well by various approximations. In general, vagueness is related to the difficulty in making sharp classification boundaries. Ambiguity is associated with one-to-many mapping. This book develops feature selection and pattern classification models which take care of these two inherent problems associated with knowledge discovery from data. Many interesting properties of the developed fuzzy-rough measures are derived and their importance from pattern classification view point is shown. Later it is shown that how neural type of learning algorithms can be integrated in fuzzy decision tree induction technique to improve their learning accuracy. Neural learning is introduced by two ways; first by transforming fuzzy decision tree structure in the equivalent Gaussian radial basis function (RBF) structure and second by directly applying back-propagation gradient descent learning algorithm on the structure of fuzzy decision trees. Former technique provide a novel solution for initialization of structure and parameters of Gaussian RBF network and later technique build novel neuro-fuzzy models known as Neuro-Fuzzy Decision Trees. It is shown that how fuzzy decision trees are functionally equivalent to Gaussian RBF networks and this equivalence is used to establish a mapping from one fuzzy decision trees to Gaussian RBF networks and vice versa. Such mappings allow applying various gradient descent type of learning algorithms to the mapped Gaussian RBF network structure and improve classification accuracy of the model. It is shown that how fuzzy decision trees' classification accuracy is improved after mapping to Gaussian RBF network and how the structure of fuzzy decision trees are changing after mapping back from Gaussian RBF network structure. This leads to an interesting discussion about destructive and non-destructive type of learning algorithms. Later it is shown that how Neuro-fuzzy decision trees are keeping the structure of fuzzy decision trees intact and still improve their learning accuracy. All the proposed algorithms have been stated explicitly in the formal notation and in pseudo code format. Extensive computational experiments have been reported and the proposed algorithms have been experimentally compared with well-known algorithms from the literature using real-world standard datasets. Readers will also find literature review of rough sets and fuzzy decision trees very useful, especially classification of fuzzy decision tree literature in six different categories, rough sets fundamentals and applications of fuzzy decision trees and rough sets theory in many domains.