Stochastic Gaussian Process Model Averaging for High-Dimensional Inputs

Maxime Xuereb, Szu Hui Ng, Giulia Pedrielli

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Many statistical learning methodologies exhibit loss of efficiency and accuracy when applied to large, high-dimensional data-sets. Such loss is exacerbated by noisy data. In this paper, we focus on Gaussian Processes (GPs), a family of non-parametric approaches used in machine learning and Bayesian Optimization. In fact, GPs show difficulty scaling with the input data size and dimensionality. This paper presents, for the first time, the Stochastic GP Model Averaging (SGPMA) algorithm, to tackle both challenges. SGPMA uses a Bayesian approach to weight several predictors, each trained with an independent subset of the initial data-set (solving the large data-sets issue), and defined in a low-dimensional embedding of the original space (solving the high dimensionality). We conduct several experiments with different input size and dimensionality. The results show that our methodology is superior to naive averaging and that the embedding choice is critical to manage the computational cost / prediction accuracy trade-off.

Original languageEnglish (US)
Title of host publicationProceedings of the 2020 Winter Simulation Conference, WSC 2020
EditorsK.-H. Bae, B. Feng, S. Kim, S. Lazarova-Molnar, Z. Zheng, T. Roeder, R. Thiesing
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages373-384
Number of pages12
ISBN (Electronic)9781728194998
DOIs
StatePublished - Dec 14 2020
Externally publishedYes
Event2020 Winter Simulation Conference, WSC 2020 - Orlando, United States
Duration: Dec 14 2020Dec 18 2020

Publication series

NameProceedings - Winter Simulation Conference
Volume2020-December
ISSN (Print)0891-7736

Conference

Conference2020 Winter Simulation Conference, WSC 2020
Country/TerritoryUnited States
CityOrlando
Period12/14/2012/18/20

ASJC Scopus subject areas

  • Software
  • Modeling and Simulation
  • Computer Science Applications

Fingerprint

Dive into the research topics of 'Stochastic Gaussian Process Model Averaging for High-Dimensional Inputs'. Together they form a unique fingerprint.

Cite this