TY - JOUR
T1 - Learning from failures of protocol in cross-cultural research
AU - Hruschka, Daniel
AU - Munira, Shirajum
AU - Jesmin, Khaleda
AU - Hackman, Joseph
AU - Tiokhin, Leonid
N1 - Funding Information:
ACKNOWLEDGMENTS. D.J.H. was supported by US National Science Foundation Grant BCS-1150813, jointly funded by Programs in Cultural Anthropology and Social Psychology Program and Decision, Risk, and Management Sciences, and Grant BCS-1658766, jointly funded by Programs in Cultural Anthropology and Methodology, Measurement, and Statistics. Additional support was provided by National Science Foundation Grants BCS-1647219 and BCS-1623555 for workshops leading up to the Sackler Colloquium.
Publisher Copyright:
© 2018 National Academy of Sciences. All rights reserved.
PY - 2018/11/6
Y1 - 2018/11/6
N2 - The many tools that social and behavioral scientists use to gather data from their fellow humans have, in most cases, been honed on a rarefied subset of humanity: highly educated participants with unique capacities, experiences, motivations, and social expectations. Through this honing process, researchers have developed protocols that extract information from these participants with great efficiency. However, as researchers reach out to broader populations, it is unclear whether these highly refined protocols are robust to cultural differences in skills, motivations, and expected modes of social interaction. In this paper, we illustrate the kinds of mismatches that can arise when using these highly refined protocols with nontypical populations by describing our experience translating an apparently simple social discounting protocol to work in rural Bangladesh. Multiple rounds of piloting and revision revealed a number of tacit assumptions about how participants should perceive, understand, and respond to key elements of the protocol. These included facility with numbers, letters, abstract number lines, and 2D geometric shapes, and the treatment of decisions as a series of isolated events. Through on-the-ground observation and a collaborative refinement process, we developed a protocol that worked both in Bangladesh and among US college students. More systematic study of the process of adapting common protocols to new contexts will provide valuable information about the range of skills, motivations, and modes of interaction that participants bring to studies as we develop a more diverse and inclusive social and behavioral science.
AB - The many tools that social and behavioral scientists use to gather data from their fellow humans have, in most cases, been honed on a rarefied subset of humanity: highly educated participants with unique capacities, experiences, motivations, and social expectations. Through this honing process, researchers have developed protocols that extract information from these participants with great efficiency. However, as researchers reach out to broader populations, it is unclear whether these highly refined protocols are robust to cultural differences in skills, motivations, and expected modes of social interaction. In this paper, we illustrate the kinds of mismatches that can arise when using these highly refined protocols with nontypical populations by describing our experience translating an apparently simple social discounting protocol to work in rural Bangladesh. Multiple rounds of piloting and revision revealed a number of tacit assumptions about how participants should perceive, understand, and respond to key elements of the protocol. These included facility with numbers, letters, abstract number lines, and 2D geometric shapes, and the treatment of decisions as a series of isolated events. Through on-the-ground observation and a collaborative refinement process, we developed a protocol that worked both in Bangladesh and among US college students. More systematic study of the process of adapting common protocols to new contexts will provide valuable information about the range of skills, motivations, and modes of interaction that participants bring to studies as we develop a more diverse and inclusive social and behavioral science.
KW - Bangladesh
KW - Cross-cultural
KW - Diversity
KW - Generalizability
KW - Social discounting
UR - http://www.scopus.com/inward/record.url?scp=85056088604&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85056088604&partnerID=8YFLogxK
U2 - 10.1073/pnas.1721166115
DO - 10.1073/pnas.1721166115
M3 - Article
C2 - 30397138
AN - SCOPUS:85056088604
SN - 0027-8424
VL - 115
SP - 11428
EP - 11434
JO - Proceedings of the National Academy of Sciences of the United States of America
JF - Proceedings of the National Academy of Sciences of the United States of America
IS - 45
ER -