Selecting a near-optimal design for multiple criteria with improved robustness to different user priorities

Sarah E. Burke, Lu Lu, Christine M. Anderson-Cook, Douglas Montgomery

Research output: Contribution to journalArticle

Abstract

In a decision-making process, relying on only one objective can often lead to oversimplified decisions that ignore important considerations. Incorporating multiple, and likely competing, objectives is critical for balancing trade-offs on different aspects of performance. When multiple objectives are considered, it is often hard to make a precise decision on how to weight the different objectives when combining their performance for ranking and selecting designs. We show that there are situations when selecting a design with near-optimality for a broad range of weight combinations of the criteria is a better test selection strategy compared with choosing a design that is strictly optimal under very restricted conditions. We propose a new design selection strategy that identifies several top-ranked solutions across broad weight combinations using layered Pareto fronts and then selects the final design that offers the best robustness to different user priorities. This method involves identifying multiple leading solutions based on the primary objectives and comparing the alternatives using secondary objectives to make the final decision. We focus on the selection of screening designs because they are widely used both in industrial research, development, and operational testing. The method is illustrated with an example of selecting a single design from a catalog of designs of a fixed size. However, the method can be adapted to more general designed experiment selection problems that involve searching through a large design space.

Original languageEnglish (US)
JournalQuality and Reliability Engineering International
DOIs
StateAccepted/In press - Jan 1 2018

Fingerprint

Optimal design
Multiple criteria
Robustness
Industrial research
Screening
Decision making
Testing
Experiments
Pareto
Multiple objectives
Optimality
Ranking
Decision-making process
Experiment
Trade-offs

Keywords

  • D-optimality
  • DMRCS
  • factor correlations
  • G-optimality
  • I-optimality
  • Pareto front
  • power
  • projections of designs

ASJC Scopus subject areas

  • Safety, Risk, Reliability and Quality
  • Management Science and Operations Research

Cite this

Selecting a near-optimal design for multiple criteria with improved robustness to different user priorities. / Burke, Sarah E.; Lu, Lu; Anderson-Cook, Christine M.; Montgomery, Douglas.

In: Quality and Reliability Engineering International, 01.01.2018.

Research output: Contribution to journalArticle

@article{735a6c36947a4590baa2d364455528af,
title = "Selecting a near-optimal design for multiple criteria with improved robustness to different user priorities",
abstract = "In a decision-making process, relying on only one objective can often lead to oversimplified decisions that ignore important considerations. Incorporating multiple, and likely competing, objectives is critical for balancing trade-offs on different aspects of performance. When multiple objectives are considered, it is often hard to make a precise decision on how to weight the different objectives when combining their performance for ranking and selecting designs. We show that there are situations when selecting a design with near-optimality for a broad range of weight combinations of the criteria is a better test selection strategy compared with choosing a design that is strictly optimal under very restricted conditions. We propose a new design selection strategy that identifies several top-ranked solutions across broad weight combinations using layered Pareto fronts and then selects the final design that offers the best robustness to different user priorities. This method involves identifying multiple leading solutions based on the primary objectives and comparing the alternatives using secondary objectives to make the final decision. We focus on the selection of screening designs because they are widely used both in industrial research, development, and operational testing. The method is illustrated with an example of selecting a single design from a catalog of designs of a fixed size. However, the method can be adapted to more general designed experiment selection problems that involve searching through a large design space.",
keywords = "D-optimality, DMRCS, factor correlations, G-optimality, I-optimality, Pareto front, power, projections of designs",
author = "Burke, {Sarah E.} and Lu Lu and Anderson-Cook, {Christine M.} and Douglas Montgomery",
year = "2018",
month = "1",
day = "1",
doi = "10.1002/qre.2413",
language = "English (US)",
journal = "Quality and Reliability Engineering International",
issn = "0748-8017",
publisher = "John Wiley and Sons Ltd",

}

TY - JOUR

T1 - Selecting a near-optimal design for multiple criteria with improved robustness to different user priorities

AU - Burke, Sarah E.

AU - Lu, Lu

AU - Anderson-Cook, Christine M.

AU - Montgomery, Douglas

PY - 2018/1/1

Y1 - 2018/1/1

N2 - In a decision-making process, relying on only one objective can often lead to oversimplified decisions that ignore important considerations. Incorporating multiple, and likely competing, objectives is critical for balancing trade-offs on different aspects of performance. When multiple objectives are considered, it is often hard to make a precise decision on how to weight the different objectives when combining their performance for ranking and selecting designs. We show that there are situations when selecting a design with near-optimality for a broad range of weight combinations of the criteria is a better test selection strategy compared with choosing a design that is strictly optimal under very restricted conditions. We propose a new design selection strategy that identifies several top-ranked solutions across broad weight combinations using layered Pareto fronts and then selects the final design that offers the best robustness to different user priorities. This method involves identifying multiple leading solutions based on the primary objectives and comparing the alternatives using secondary objectives to make the final decision. We focus on the selection of screening designs because they are widely used both in industrial research, development, and operational testing. The method is illustrated with an example of selecting a single design from a catalog of designs of a fixed size. However, the method can be adapted to more general designed experiment selection problems that involve searching through a large design space.

AB - In a decision-making process, relying on only one objective can often lead to oversimplified decisions that ignore important considerations. Incorporating multiple, and likely competing, objectives is critical for balancing trade-offs on different aspects of performance. When multiple objectives are considered, it is often hard to make a precise decision on how to weight the different objectives when combining their performance for ranking and selecting designs. We show that there are situations when selecting a design with near-optimality for a broad range of weight combinations of the criteria is a better test selection strategy compared with choosing a design that is strictly optimal under very restricted conditions. We propose a new design selection strategy that identifies several top-ranked solutions across broad weight combinations using layered Pareto fronts and then selects the final design that offers the best robustness to different user priorities. This method involves identifying multiple leading solutions based on the primary objectives and comparing the alternatives using secondary objectives to make the final decision. We focus on the selection of screening designs because they are widely used both in industrial research, development, and operational testing. The method is illustrated with an example of selecting a single design from a catalog of designs of a fixed size. However, the method can be adapted to more general designed experiment selection problems that involve searching through a large design space.

KW - D-optimality

KW - DMRCS

KW - factor correlations

KW - G-optimality

KW - I-optimality

KW - Pareto front

KW - power

KW - projections of designs

UR - http://www.scopus.com/inward/record.url?scp=85055686487&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85055686487&partnerID=8YFLogxK

U2 - 10.1002/qre.2413

DO - 10.1002/qre.2413

M3 - Article

JO - Quality and Reliability Engineering International

JF - Quality and Reliability Engineering International

SN - 0748-8017

ER -