Using simulated annealing to optimize the feature selection problem in marketing applications

Ronen Meiri, Jacob Zahavi

Research output: Contribution to journalArticlepeer-review


The feature selection (also, specification) problem is concerned with finding the most influential subset of predictors in predictive modeling from a much larger set of potential predictors that can contain hundreds of predictors. The problem belongs to the realm of combinatorial optimization where the objective is to find the subset of variables that optimize the value of some goodness of fit function. Due to the dimensionality of the problem, the feature selection problem belongs to the group of NP-hard problems. Most of the available predictors are noisy or redundant and add very little, if any, to the prediction power of the model. Using all the predictors in the model often results in strong over-fitting and very poor predictions. Constructing a prediction model by checking out all possible subsets is impractical due to computational volume. Looking on the contribution of each predictor separately is not accurate because it ignores the inter-correlations between predictors. As a result, no analytic solution is available for the feature selection problem, requiring that one resorts to heuristics. In this paper we employ the simulated annealing (SA) approach, which is one of the leading stochastic search methods, for specifying a large-scale linear regression model. The SA results are compared to the results of the more common stepwise regression (SWR) approach for model specification. The models are applied on realistic data sets in database marketing. We also use simulated data sets to investigate what data characteristics make the SWR approach equivalent to the supposedly more superior SA approach.

Original languageEnglish
Pages (from-to)842-858
Number of pages17
JournalEuropean Journal of Operational Research
Issue number3
StatePublished - 16 Jun 2006


  • Database marketing
  • Feature selection
  • Simulated annealing
  • Stepwise regression


Dive into the research topics of 'Using simulated annealing to optimize the feature selection problem in marketing applications'. Together they form a unique fingerprint.

Cite this