Using more data to speed-up training time

Shai Shalev-Shwartz, Ohad Shamir, Eran Tromer

Research output: Contribution to journalConference articlepeer-review

Abstract

In many recent applications, data is plentiful. By now, we have a rather clear understanding of how more data can be used to improve the accuracy of learning algorithms. Recently, there has been a growing interest in understanding how more data can be leveraged to reduce the required training runtime. In this paper, we study the runtime of learning as a function of the number of available training examples, and underscore the main highlevel techniques. We provide the first formal positive result showing that even in the unrealizable case, the runtime can decrease exponentially while only requiring a polynomial growth of the number of examples. Our construction corresponds to a synthetic learning problem and an interesting open question is whether the tradeoff can be shown for more natural learning problems. We spell out several interesting candidates of natural learning problems for which we conjecture that there is a tradeoff between computational and sample complexity.

Original languageEnglish
Pages (from-to)1019-1027
Number of pages9
JournalJournal of Machine Learning Research
Volume22
StatePublished - 2012
Event15th International Conference on Artificial Intelligence and Statistics, AISTATS 2012 - La Palma, Spain
Duration: 21 Apr 201223 Apr 2012

Fingerprint

Dive into the research topics of 'Using more data to speed-up training time'. Together they form a unique fingerprint.

Cite this