Regularization in regression with bounded noise: A Chebyshev center approach

Amir Beck*, Yonina C. Eldar

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

We consider the problem of estimating a vector z in the regression model b = Az+w, where w is an unknown but bounded noise. As in many regularization schemes, we assume that an upper bound on the norm of z is available. To estimate z we propose a relaxation of the Chebyshev center, which is the vector that minimizes the worst-case estimation error over all feasible vectors z. Relying on recent results regarding strong duality of noncouvex quadratic optimization problems with two quadratic constraints, we prove that in the complex domain our approach leads to the exact Chebyshev center. In the real domain, this strategy results in a "pretty good" approximation of the true Chebyshev center. As we show, our estimate can be viewed as a Tikhonov regularization with a special choice of parameter that can be found efficiently by solving a convex optimization problem with two variables or a semidefinite program with three variables, regardless of the problem size. When the norm constraint on z is a Euclidean one, the problem reduces to a single-variable convex minimization problem. We then demonstrate via numerical examples that our estimator can outperform other conventional methods, such as least-squares and regularized least-squares, with respect to the estimation error. Finally, we extend our methodology to other feasible parameter sets, showing that the total least-squares (TLS) and regularized TLS can be obtained as special cases of our general approach.

Original languageEnglish
Pages (from-to)606-625
Number of pages20
JournalSIAM Journal on Matrix Analysis and Applications
Volume29
Issue number2
DOIs
StatePublished - 2007
Externally publishedYes

Keywords

  • Bounded error estimation
  • Chebyshev center
  • Strong duality
  • Uoncouvex quadratic optimization

Fingerprint

Dive into the research topics of 'Regularization in regression with bounded noise: A Chebyshev center approach'. Together they form a unique fingerprint.

Cite this