Tractable sampling strategies for quantile-based ordinal optimization

Dongwook Shin, Mark Broadie, Assaf Zeevi

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review


This paper describes and analyzes the problem of selecting the best of several alternatives ("systems"), where they are compared based on quantiles of their performances. The quantiles cannot be evaluated analytically but it is possible to sequentially sample from each system. The objective is to dynamically allocate a finite sampling budget to minimize the probability of falsely selecting non-best systems. To formulate this problem in a tractable form, we introduce an objective associated with the probability of false selection using large deviations theory and leverage it to design well-performing dynamic sampling policies. We first propose a naive policy that optimizes the aforementioned objective when the sampling budget is sufficiently large. We introduce two variants of the naive policy with the aim of improving finite-Time performance; these policies retain the asymptotic performance of the naive one in some cases, while dramatically improving its finite-Time performance.

Original languageEnglish
Title of host publication2016 Winter Simulation Conference
Subtitle of host publicationSimulating Complex Service Systems, WSC 2016
EditorsTheresa M. Roeder, Peter I. Frazier, Robert Szechtman, Enlu Zhou
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages12
ISBN (Electronic)9781509044863
StatePublished - 2 Jul 2016
Externally publishedYes
Event2016 Winter Simulation Conference, WSC 2016 - Arlington, United States
Duration: 11 Dec 201614 Dec 2016

Publication series

NameProceedings - Winter Simulation Conference
ISSN (Print)0891-7736


Conference2016 Winter Simulation Conference, WSC 2016
Country/TerritoryUnited States


Dive into the research topics of 'Tractable sampling strategies for quantile-based ordinal optimization'. Together they form a unique fingerprint.

Cite this