ϵ-Equivalence of Feature Selection Rules

The ranking of features according to a feature selection criteria is examined. The concept of ϵ-equivalence is introduced to measure the extent to which a ranking deviates from the ranking induced by the probability of error rule. The relationship between the ϵ-equivalence of a given rule and the bounds on the probability of error derived from this rule is demonstrated. Illustrations of the ϵ-equivalence concept are presented for Shannon's equivocation rule, the quadratic equivocation rule, and the Bhattacharyya rule. A numerical example concludes the presentation.