Learning from failures of protocol in cross-cultural research

Daniel Hruschka, Shirajum Munira, Khaleda Jesmin, Joseph Hackman, Leonid Tiokhin

Research output: Contribution to journalArticle

5 Citations (Scopus)

Abstract

The many tools that social and behavioral scientists use to gather data from their fellow humans have, in most cases, been honed on a rarefied subset of humanity: highly educated participants with unique capacities, experiences, motivations, and social expectations. Through this honing process, researchers have developed protocols that extract information from these participants with great efficiency. However, as researchers reach out to broader populations, it is unclear whether these highly refined protocols are robust to cultural differences in skills, motivations, and expected modes of social interaction. In this paper, we illustrate the kinds of mismatches that can arise when using these highly refined protocols with nontypical populations by describing our experience translating an apparently simple social discounting protocol to work in rural Bangladesh. Multiple rounds of piloting and revision revealed a number of tacit assumptions about how participants should perceive, understand, and respond to key elements of the protocol. These included facility with numbers, letters, abstract number lines, and 2D geometric shapes, and the treatment of decisions as a series of isolated events. Through on-the-ground observation and a collaborative refinement process, we developed a protocol that worked both in Bangladesh and among US college students. More systematic study of the process of adapting common protocols to new contexts will provide valuable information about the range of skills, motivations, and modes of interaction that participants bring to studies as we develop a more diverse and inclusive social and behavioral science.

Original languageEnglish (US)
Pages (from-to)11428-11434
Number of pages7
JournalProceedings of the National Academy of Sciences of the United States of America
Volume115
Issue number45
DOIs
StatePublished - Nov 6 2018

Fingerprint

Motivation
Bangladesh
Learning
Research
Research Personnel
Behavioral Sciences
Social Sciences
Interpersonal Relations
Population
Observation
Students
Efficiency
Therapeutics

Keywords

  • Bangladesh
  • Cross-cultural
  • Diversity
  • Generalizability
  • Social discounting

ASJC Scopus subject areas

  • General

Cite this

Learning from failures of protocol in cross-cultural research. / Hruschka, Daniel; Munira, Shirajum; Jesmin, Khaleda; Hackman, Joseph; Tiokhin, Leonid.

In: Proceedings of the National Academy of Sciences of the United States of America, Vol. 115, No. 45, 06.11.2018, p. 11428-11434.

Research output: Contribution to journalArticle

Hruschka, Daniel ; Munira, Shirajum ; Jesmin, Khaleda ; Hackman, Joseph ; Tiokhin, Leonid. / Learning from failures of protocol in cross-cultural research. In: Proceedings of the National Academy of Sciences of the United States of America. 2018 ; Vol. 115, No. 45. pp. 11428-11434.
@article{7c167c00a5dd46c2bdf4959e44e65370,
title = "Learning from failures of protocol in cross-cultural research",
abstract = "The many tools that social and behavioral scientists use to gather data from their fellow humans have, in most cases, been honed on a rarefied subset of humanity: highly educated participants with unique capacities, experiences, motivations, and social expectations. Through this honing process, researchers have developed protocols that extract information from these participants with great efficiency. However, as researchers reach out to broader populations, it is unclear whether these highly refined protocols are robust to cultural differences in skills, motivations, and expected modes of social interaction. In this paper, we illustrate the kinds of mismatches that can arise when using these highly refined protocols with nontypical populations by describing our experience translating an apparently simple social discounting protocol to work in rural Bangladesh. Multiple rounds of piloting and revision revealed a number of tacit assumptions about how participants should perceive, understand, and respond to key elements of the protocol. These included facility with numbers, letters, abstract number lines, and 2D geometric shapes, and the treatment of decisions as a series of isolated events. Through on-the-ground observation and a collaborative refinement process, we developed a protocol that worked both in Bangladesh and among US college students. More systematic study of the process of adapting common protocols to new contexts will provide valuable information about the range of skills, motivations, and modes of interaction that participants bring to studies as we develop a more diverse and inclusive social and behavioral science.",
keywords = "Bangladesh, Cross-cultural, Diversity, Generalizability, Social discounting",
author = "Daniel Hruschka and Shirajum Munira and Khaleda Jesmin and Joseph Hackman and Leonid Tiokhin",
year = "2018",
month = "11",
day = "6",
doi = "10.1073/pnas.1721166115",
language = "English (US)",
volume = "115",
pages = "11428--11434",
journal = "Proceedings of the National Academy of Sciences of the United States of America",
issn = "0027-8424",
number = "45",

}

TY - JOUR

T1 - Learning from failures of protocol in cross-cultural research

AU - Hruschka, Daniel

AU - Munira, Shirajum

AU - Jesmin, Khaleda

AU - Hackman, Joseph

AU - Tiokhin, Leonid

PY - 2018/11/6

Y1 - 2018/11/6

N2 - The many tools that social and behavioral scientists use to gather data from their fellow humans have, in most cases, been honed on a rarefied subset of humanity: highly educated participants with unique capacities, experiences, motivations, and social expectations. Through this honing process, researchers have developed protocols that extract information from these participants with great efficiency. However, as researchers reach out to broader populations, it is unclear whether these highly refined protocols are robust to cultural differences in skills, motivations, and expected modes of social interaction. In this paper, we illustrate the kinds of mismatches that can arise when using these highly refined protocols with nontypical populations by describing our experience translating an apparently simple social discounting protocol to work in rural Bangladesh. Multiple rounds of piloting and revision revealed a number of tacit assumptions about how participants should perceive, understand, and respond to key elements of the protocol. These included facility with numbers, letters, abstract number lines, and 2D geometric shapes, and the treatment of decisions as a series of isolated events. Through on-the-ground observation and a collaborative refinement process, we developed a protocol that worked both in Bangladesh and among US college students. More systematic study of the process of adapting common protocols to new contexts will provide valuable information about the range of skills, motivations, and modes of interaction that participants bring to studies as we develop a more diverse and inclusive social and behavioral science.

AB - The many tools that social and behavioral scientists use to gather data from their fellow humans have, in most cases, been honed on a rarefied subset of humanity: highly educated participants with unique capacities, experiences, motivations, and social expectations. Through this honing process, researchers have developed protocols that extract information from these participants with great efficiency. However, as researchers reach out to broader populations, it is unclear whether these highly refined protocols are robust to cultural differences in skills, motivations, and expected modes of social interaction. In this paper, we illustrate the kinds of mismatches that can arise when using these highly refined protocols with nontypical populations by describing our experience translating an apparently simple social discounting protocol to work in rural Bangladesh. Multiple rounds of piloting and revision revealed a number of tacit assumptions about how participants should perceive, understand, and respond to key elements of the protocol. These included facility with numbers, letters, abstract number lines, and 2D geometric shapes, and the treatment of decisions as a series of isolated events. Through on-the-ground observation and a collaborative refinement process, we developed a protocol that worked both in Bangladesh and among US college students. More systematic study of the process of adapting common protocols to new contexts will provide valuable information about the range of skills, motivations, and modes of interaction that participants bring to studies as we develop a more diverse and inclusive social and behavioral science.

KW - Bangladesh

KW - Cross-cultural

KW - Diversity

KW - Generalizability

KW - Social discounting

UR - http://www.scopus.com/inward/record.url?scp=85056088604&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85056088604&partnerID=8YFLogxK

U2 - 10.1073/pnas.1721166115

DO - 10.1073/pnas.1721166115

M3 - Article

VL - 115

SP - 11428

EP - 11434

JO - Proceedings of the National Academy of Sciences of the United States of America

JF - Proceedings of the National Academy of Sciences of the United States of America

SN - 0027-8424

IS - 45

ER -