A Weighted Multi-k Extension of ReliefF for Robust Feature Selection

This research proposes an improved version of ReliefF method for feature selection tasks in data mining. ReliefF-based algorithms are versatile and effective feature evaluators that identify conditional dependencies among features across instances, and they are commonly used in the preprocessing stage for tasks such as classification and regression. Since its introduction, numerous extensions of the ReliefF approach have been proposed to address redundancy, irrelevance, and noisy features, as well as the inherent limitations of the model in handling both binary and multi-class datasets. In this paper, we examine the effect of the usage of multiple number of neighbors in ReliefF algorithm concurrently with the help of standard majority voting and weighted majority voting strategies. The proposed method, is called Multi-k Weighted ReliefF (MW-ReliefF). It requires a set of different number of neighbors to calculate near hit and near miss values for each. Then it updates those values with respect to the weights of the predetermined number of nearest neighbors. The method is implemented as a preprocessing step for feature reduction before classifying ۲۰ benchmark binary and multi-class datasets and classification performance results are obtained and compared to the counterparts of ReliefF method and its traditional variants in order to prove the validity of MW-ReliefF. In fact, the experimental findings demonstrated that the proposed algorithm outperformed the original ReliefF methods across almost all datasets.

کلیدواژه ها:

Feature selection ، ReliefF ، Classification ، Data mining ، Machine Learning