Abstract

Problem: This case study involves a mixture experiment with three components where the primary response, fragrance intensity, is subjectively evaluated by 40 fragrance experts using a seven-point intensity scale. The problem is motivated by the addition of two new components that are purported to enhance fragrance intensity. The common practice in analyzing ordinal ratings is to use the averaged ratings across panelists and to analyze these responses as if they were numeric. However, this approach yields results that are not meaningful and difficult to interpret because of the ordinality of the rating scale. Recommendations for modeling ordinal responses from mixture experiments have not been dealt with in the literature, so the techniques described here have a broader application to other experimental settings. Approach: When responses are subjectively assessed and the rating scale is arbitrarily assigned, Anderson's (1984) stereotype regression model presents a flexible alternative to the widely popular proportional odds model. The modeling approach presented in this paper addresses the issue of inestimability of model parameters due to perfect collinearity among the mixture components, as well as the uncertainty in the discretization of the ordered, categorical response. For this case study, the assignment of ordinal scales to sensory levels is done in an arbitrary manner, such that no historical studies support the effectiveness of the assignment. Comparisons are made among the stereotype, the proportional odds, and traditional regression modeling where the numeric averages of the ordinal responses are analyzed. Results: In this case study, the stereotype model revealed that adjacent categories of the ordered, categorical response are indistinguishable with respect to the changes in the proportions of the mixture components. Internal tests for indistinguishability provided statistical justification for collapsing the categories, and in so doing, the more parsimonious proportional odds form was found to satisfactorily model and improve the interpretability of the reconfigured responses. Analysis revealed that one of the two new components introduced to the formula significantly improved the perception of fragrance intensity; hence, adoption of the new compound was strongly justified.

Original languageEnglish (US)
Pages (from-to)196-208
Number of pages13
JournalJournal of Quality Technology
Volume48
Issue number2
DOIs
StatePublished - Jan 1 2016

Fingerprint

Fragrances
Experiments
Experiment
Stereotypes
Modeling

Keywords

  • Categorical Data
  • Formulation Experiments
  • Proportional Odds
  • Rated Responses
  • Stereotype

ASJC Scopus subject areas

  • Safety, Risk, Reliability and Quality
  • Strategy and Management
  • Management Science and Operations Research
  • Industrial and Manufacturing Engineering

Cite this

Analysis of subjective ordinal responses in mixture experiments. / Mancenido, Michelle; Pan, Rong; Montgomery, Douglas.

In: Journal of Quality Technology, Vol. 48, No. 2, 01.01.2016, p. 196-208.

Research output: Contribution to journalArticle

@article{73743f501df24d01ac6daaec677949b4,
title = "Analysis of subjective ordinal responses in mixture experiments",
abstract = "Problem: This case study involves a mixture experiment with three components where the primary response, fragrance intensity, is subjectively evaluated by 40 fragrance experts using a seven-point intensity scale. The problem is motivated by the addition of two new components that are purported to enhance fragrance intensity. The common practice in analyzing ordinal ratings is to use the averaged ratings across panelists and to analyze these responses as if they were numeric. However, this approach yields results that are not meaningful and difficult to interpret because of the ordinality of the rating scale. Recommendations for modeling ordinal responses from mixture experiments have not been dealt with in the literature, so the techniques described here have a broader application to other experimental settings. Approach: When responses are subjectively assessed and the rating scale is arbitrarily assigned, Anderson's (1984) stereotype regression model presents a flexible alternative to the widely popular proportional odds model. The modeling approach presented in this paper addresses the issue of inestimability of model parameters due to perfect collinearity among the mixture components, as well as the uncertainty in the discretization of the ordered, categorical response. For this case study, the assignment of ordinal scales to sensory levels is done in an arbitrary manner, such that no historical studies support the effectiveness of the assignment. Comparisons are made among the stereotype, the proportional odds, and traditional regression modeling where the numeric averages of the ordinal responses are analyzed. Results: In this case study, the stereotype model revealed that adjacent categories of the ordered, categorical response are indistinguishable with respect to the changes in the proportions of the mixture components. Internal tests for indistinguishability provided statistical justification for collapsing the categories, and in so doing, the more parsimonious proportional odds form was found to satisfactorily model and improve the interpretability of the reconfigured responses. Analysis revealed that one of the two new components introduced to the formula significantly improved the perception of fragrance intensity; hence, adoption of the new compound was strongly justified.",
keywords = "Categorical Data, Formulation Experiments, Proportional Odds, Rated Responses, Stereotype",
author = "Michelle Mancenido and Rong Pan and Douglas Montgomery",
year = "2016",
month = "1",
day = "1",
doi = "10.1080/00224065.2016.11918159",
language = "English (US)",
volume = "48",
pages = "196--208",
journal = "Journal of Quality Technology",
issn = "0022-4065",
publisher = "American Society for Quality",
number = "2",

}

TY - JOUR

T1 - Analysis of subjective ordinal responses in mixture experiments

AU - Mancenido, Michelle

AU - Pan, Rong

AU - Montgomery, Douglas

PY - 2016/1/1

Y1 - 2016/1/1

N2 - Problem: This case study involves a mixture experiment with three components where the primary response, fragrance intensity, is subjectively evaluated by 40 fragrance experts using a seven-point intensity scale. The problem is motivated by the addition of two new components that are purported to enhance fragrance intensity. The common practice in analyzing ordinal ratings is to use the averaged ratings across panelists and to analyze these responses as if they were numeric. However, this approach yields results that are not meaningful and difficult to interpret because of the ordinality of the rating scale. Recommendations for modeling ordinal responses from mixture experiments have not been dealt with in the literature, so the techniques described here have a broader application to other experimental settings. Approach: When responses are subjectively assessed and the rating scale is arbitrarily assigned, Anderson's (1984) stereotype regression model presents a flexible alternative to the widely popular proportional odds model. The modeling approach presented in this paper addresses the issue of inestimability of model parameters due to perfect collinearity among the mixture components, as well as the uncertainty in the discretization of the ordered, categorical response. For this case study, the assignment of ordinal scales to sensory levels is done in an arbitrary manner, such that no historical studies support the effectiveness of the assignment. Comparisons are made among the stereotype, the proportional odds, and traditional regression modeling where the numeric averages of the ordinal responses are analyzed. Results: In this case study, the stereotype model revealed that adjacent categories of the ordered, categorical response are indistinguishable with respect to the changes in the proportions of the mixture components. Internal tests for indistinguishability provided statistical justification for collapsing the categories, and in so doing, the more parsimonious proportional odds form was found to satisfactorily model and improve the interpretability of the reconfigured responses. Analysis revealed that one of the two new components introduced to the formula significantly improved the perception of fragrance intensity; hence, adoption of the new compound was strongly justified.

AB - Problem: This case study involves a mixture experiment with three components where the primary response, fragrance intensity, is subjectively evaluated by 40 fragrance experts using a seven-point intensity scale. The problem is motivated by the addition of two new components that are purported to enhance fragrance intensity. The common practice in analyzing ordinal ratings is to use the averaged ratings across panelists and to analyze these responses as if they were numeric. However, this approach yields results that are not meaningful and difficult to interpret because of the ordinality of the rating scale. Recommendations for modeling ordinal responses from mixture experiments have not been dealt with in the literature, so the techniques described here have a broader application to other experimental settings. Approach: When responses are subjectively assessed and the rating scale is arbitrarily assigned, Anderson's (1984) stereotype regression model presents a flexible alternative to the widely popular proportional odds model. The modeling approach presented in this paper addresses the issue of inestimability of model parameters due to perfect collinearity among the mixture components, as well as the uncertainty in the discretization of the ordered, categorical response. For this case study, the assignment of ordinal scales to sensory levels is done in an arbitrary manner, such that no historical studies support the effectiveness of the assignment. Comparisons are made among the stereotype, the proportional odds, and traditional regression modeling where the numeric averages of the ordinal responses are analyzed. Results: In this case study, the stereotype model revealed that adjacent categories of the ordered, categorical response are indistinguishable with respect to the changes in the proportions of the mixture components. Internal tests for indistinguishability provided statistical justification for collapsing the categories, and in so doing, the more parsimonious proportional odds form was found to satisfactorily model and improve the interpretability of the reconfigured responses. Analysis revealed that one of the two new components introduced to the formula significantly improved the perception of fragrance intensity; hence, adoption of the new compound was strongly justified.

KW - Categorical Data

KW - Formulation Experiments

KW - Proportional Odds

KW - Rated Responses

KW - Stereotype

UR - http://www.scopus.com/inward/record.url?scp=85023602326&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85023602326&partnerID=8YFLogxK

U2 - 10.1080/00224065.2016.11918159

DO - 10.1080/00224065.2016.11918159

M3 - Article

AN - SCOPUS:85023602326

VL - 48

SP - 196

EP - 208

JO - Journal of Quality Technology

JF - Journal of Quality Technology

SN - 0022-4065

IS - 2

ER -