Agnostic boosting

Shai Ben-David, Philip M. Long, Yishay Mansour

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

We extend the boosting paradigm to the realistic setting of agnostic learning, that is, to a setting where the training sample is generated by an arbitrary (unknown) probability distribution over examples and labels. We define a β-weak agnostic learner with respect to a hypothesis class F as follows: given a distribution P it outputs some hypothesis h ∈ F whose error is at most erP (F) + β, where erP (F) is the minimal error of an hypothesis from F under the distribution P (note that for some distributions the bound may exceed a half). We show a boosting algorithm that using the weak agnostic learner computes a hypothesis whose error is at most max{c1(β)er(F)c2(β), ε}, in time polynomial in 1/ ε. While this generalization guarantee is significantly weaker than the one resulting from the known PAC boosting algorithms, one should note that the assumption required for β-weak agnostic learner is much weaker. In fact, an important virtue of the notion of weak agnostic learning is that in many cases such learning is achieved by efficient algorithms.

Original languageEnglish
Title of host publicationComputational Learning Theory - 14th Annual Conference on Computational Learning Theory, COLT 2001 and 5th European Conference on Computational Learning Theory, EuroCOLT 2001, Proceedings
EditorsDavid Helmbold, Bob Williamson
PublisherSpringer Verlag
Pages507-516
Number of pages10
ISBN (Print)9783540423430
DOIs
StatePublished - 2001
Event14th Annual Conference on Computational Learning Theory, COLT 2001 and 5th European Conference on Computational Learning Theory, EuroCOLT 2001 - Amsterdam, Netherlands
Duration: 16 Jul 200119 Jul 2001

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume2111
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference14th Annual Conference on Computational Learning Theory, COLT 2001 and 5th European Conference on Computational Learning Theory, EuroCOLT 2001
Country/TerritoryNetherlands
CityAmsterdam
Period16/07/0119/07/01

Fingerprint

Dive into the research topics of 'Agnostic boosting'. Together they form a unique fingerprint.

Cite this