Robust universal inference

Amichai Painsky*, Meir Feder

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

Learning and making inference from a finite set of samples are among the fundamental problems in science. In most popular applications, the paradigmatic approach is to seek a model that best explains the data. This approach has many desirable properties when the number of samples is large. However, in many practical setups, data acquisition is costly and only a limited number of samples is available. In this work, we study an alternative approach for this challenging setup. Our framework suggests that the role of the train-set is not to provide a single estimated model, which may be inaccurate due to the limited number of samples. Instead, we define a class of “reasonable” models. Then, the worst-case performance in the class is controlled by a minimax estimator with respect to it. Further, we introduce a robust estimation scheme that provides minimax guarantees, also for the case where the true model is not a member of the model class. Our results draw important connections to universal prediction, the redundancy-capacity theorem, and channel capacity theory. We demonstrate our suggested scheme in different setups, showing a significant improvement in worst-case performance over currently known alternatives.

Original languageEnglish
Article number773
JournalEntropy
Volume23
Issue number6
DOIs
StatePublished - Jun 2021

Keywords

  • Estimation theory
  • Minimax estimation
  • Minimax risk
  • Statistical inference
  • Universal prediction

Fingerprint

Dive into the research topics of 'Robust universal inference'. Together they form a unique fingerprint.

Cite this