Empirical Analysis of the performance of variance estimators in sequential single-run ranking & selection

The case of time dilation algorithm

Giulia Pedrielli, Yinchao Zhu, Loo Hay Lee, Haobin Li

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

Ranking and Selection has acquired an important role in the Simulation-Optimization field, where the different alternatives can be evaluated by discrete event simulation (DES). Black box approaches have dominated the literature by interpreting the DES as an oracle providing i.i.d. observations. Another relevant family of algorithms, instead, runs each simulator once and observes time series. This paper focuses on such a method, Time Dilation with Optimal Computing Budget Allocation (TD-OCBA), recently developed by the authors. One critical aspect of TD-OCBA is estimating the response given correlated observations. In this paper, we are specifically concerned with the estimator of the variance of the response which plays a crucial role in simulation budget allocation. We propose an empirical analysis over the performance impact on TD-OCBA of several variance estimators involved in resource allocation. Their performances are discussed in the typical probability of correct selection (PCS) framework.

Original languageEnglish (US)
Title of host publication2016 Winter Simulation Conference: Simulating Complex Service Systems, WSC 2016
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages738-748
Number of pages11
ISBN (Electronic)9781509044863
DOIs
StatePublished - Jan 17 2017
Externally publishedYes
Event2016 Winter Simulation Conference, WSC 2016 - Arlington, United States
Duration: Dec 11 2016Dec 14 2016

Other

Other2016 Winter Simulation Conference, WSC 2016
CountryUnited States
CityArlington
Period12/11/1612/14/16

Fingerprint

Variance Estimator
Empirical Analysis
Discrete event simulation
Dilation
Ranking
Discrete Event Simulation
Resource allocation
Computing
Time series
Simulators
Probability of Correct Selection
Correlated Observations
Ranking and Selection
Simulation Optimization
Black Box
Resource Allocation
Simulator
Estimator
Alternatives
Simulation

ASJC Scopus subject areas

  • Software
  • Modeling and Simulation
  • Computer Science Applications

Cite this

Pedrielli, G., Zhu, Y., Lee, L. H., & Li, H. (2017). Empirical Analysis of the performance of variance estimators in sequential single-run ranking & selection: The case of time dilation algorithm. In 2016 Winter Simulation Conference: Simulating Complex Service Systems, WSC 2016 (pp. 738-748). [7822137] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/WSC.2016.7822137

Empirical Analysis of the performance of variance estimators in sequential single-run ranking & selection : The case of time dilation algorithm. / Pedrielli, Giulia; Zhu, Yinchao; Lee, Loo Hay; Li, Haobin.

2016 Winter Simulation Conference: Simulating Complex Service Systems, WSC 2016. Institute of Electrical and Electronics Engineers Inc., 2017. p. 738-748 7822137.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Pedrielli, G, Zhu, Y, Lee, LH & Li, H 2017, Empirical Analysis of the performance of variance estimators in sequential single-run ranking & selection: The case of time dilation algorithm. in 2016 Winter Simulation Conference: Simulating Complex Service Systems, WSC 2016., 7822137, Institute of Electrical and Electronics Engineers Inc., pp. 738-748, 2016 Winter Simulation Conference, WSC 2016, Arlington, United States, 12/11/16. https://doi.org/10.1109/WSC.2016.7822137
Pedrielli G, Zhu Y, Lee LH, Li H. Empirical Analysis of the performance of variance estimators in sequential single-run ranking & selection: The case of time dilation algorithm. In 2016 Winter Simulation Conference: Simulating Complex Service Systems, WSC 2016. Institute of Electrical and Electronics Engineers Inc. 2017. p. 738-748. 7822137 https://doi.org/10.1109/WSC.2016.7822137
Pedrielli, Giulia ; Zhu, Yinchao ; Lee, Loo Hay ; Li, Haobin. / Empirical Analysis of the performance of variance estimators in sequential single-run ranking & selection : The case of time dilation algorithm. 2016 Winter Simulation Conference: Simulating Complex Service Systems, WSC 2016. Institute of Electrical and Electronics Engineers Inc., 2017. pp. 738-748
@inproceedings{0d0783821aec451b93b85d4fcb5d3e03,
title = "Empirical Analysis of the performance of variance estimators in sequential single-run ranking & selection: The case of time dilation algorithm",
abstract = "Ranking and Selection has acquired an important role in the Simulation-Optimization field, where the different alternatives can be evaluated by discrete event simulation (DES). Black box approaches have dominated the literature by interpreting the DES as an oracle providing i.i.d. observations. Another relevant family of algorithms, instead, runs each simulator once and observes time series. This paper focuses on such a method, Time Dilation with Optimal Computing Budget Allocation (TD-OCBA), recently developed by the authors. One critical aspect of TD-OCBA is estimating the response given correlated observations. In this paper, we are specifically concerned with the estimator of the variance of the response which plays a crucial role in simulation budget allocation. We propose an empirical analysis over the performance impact on TD-OCBA of several variance estimators involved in resource allocation. Their performances are discussed in the typical probability of correct selection (PCS) framework.",
author = "Giulia Pedrielli and Yinchao Zhu and Lee, {Loo Hay} and Haobin Li",
year = "2017",
month = "1",
day = "17",
doi = "10.1109/WSC.2016.7822137",
language = "English (US)",
pages = "738--748",
booktitle = "2016 Winter Simulation Conference: Simulating Complex Service Systems, WSC 2016",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
address = "United States",

}

TY - GEN

T1 - Empirical Analysis of the performance of variance estimators in sequential single-run ranking & selection

T2 - The case of time dilation algorithm

AU - Pedrielli, Giulia

AU - Zhu, Yinchao

AU - Lee, Loo Hay

AU - Li, Haobin

PY - 2017/1/17

Y1 - 2017/1/17

N2 - Ranking and Selection has acquired an important role in the Simulation-Optimization field, where the different alternatives can be evaluated by discrete event simulation (DES). Black box approaches have dominated the literature by interpreting the DES as an oracle providing i.i.d. observations. Another relevant family of algorithms, instead, runs each simulator once and observes time series. This paper focuses on such a method, Time Dilation with Optimal Computing Budget Allocation (TD-OCBA), recently developed by the authors. One critical aspect of TD-OCBA is estimating the response given correlated observations. In this paper, we are specifically concerned with the estimator of the variance of the response which plays a crucial role in simulation budget allocation. We propose an empirical analysis over the performance impact on TD-OCBA of several variance estimators involved in resource allocation. Their performances are discussed in the typical probability of correct selection (PCS) framework.

AB - Ranking and Selection has acquired an important role in the Simulation-Optimization field, where the different alternatives can be evaluated by discrete event simulation (DES). Black box approaches have dominated the literature by interpreting the DES as an oracle providing i.i.d. observations. Another relevant family of algorithms, instead, runs each simulator once and observes time series. This paper focuses on such a method, Time Dilation with Optimal Computing Budget Allocation (TD-OCBA), recently developed by the authors. One critical aspect of TD-OCBA is estimating the response given correlated observations. In this paper, we are specifically concerned with the estimator of the variance of the response which plays a crucial role in simulation budget allocation. We propose an empirical analysis over the performance impact on TD-OCBA of several variance estimators involved in resource allocation. Their performances are discussed in the typical probability of correct selection (PCS) framework.

UR - http://www.scopus.com/inward/record.url?scp=85014249508&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85014249508&partnerID=8YFLogxK

U2 - 10.1109/WSC.2016.7822137

DO - 10.1109/WSC.2016.7822137

M3 - Conference contribution

SP - 738

EP - 748

BT - 2016 Winter Simulation Conference: Simulating Complex Service Systems, WSC 2016

PB - Institute of Electrical and Electronics Engineers Inc.

ER -