Generalizability is not optional

Insights from a cross-cultural study of social discounting

Leonid Tiokhin, Joseph Hackman, Shirajum Munira, Khaleda Jesmin, Daniel Hruschka

Research output: Contribution to journalArticle

Abstract

Current scientific reforms focus more on solutions to the problem of reliability (e.g. direct replications) than generalizability. Here, we use a cross-cultural study of social discounting to illustrate the utility of a complementary focus on generalizability across diverse human populations. Social discounting is the tendency to sacrifice more for socially close individuals-a phenomenon replicated across countries and laboratories. Yet, when adapting a typical protocol to low-literacy, resource-scarce settings in Bangladesh and Indonesia, we find no independent effect of social distance on generosity, despite still documenting this effect among US participants. Several reliability and validity checks suggest that methodological issues alone cannot explain this finding. These results illustrate why we must complement replication efforts with investment in strong checks on generalizability. By failing to do so, we risk developing theories of human nature that reliably explain behaviour among only a thin slice of humanity.

Original languageEnglish (US)
Article number181386
JournalRoyal Society Open Science
Volume6
Issue number2
DOIs
StatePublished - Feb 1 2019

Fingerprint

cultural studies
social distance
Bangladesh
Indonesia
literacy
reform
resources

Keywords

  • Cross-cultural differences
  • Generalizability
  • Replication
  • Sample diversity
  • Social discounting

ASJC Scopus subject areas

  • General

Cite this

Generalizability is not optional : Insights from a cross-cultural study of social discounting. / Tiokhin, Leonid; Hackman, Joseph; Munira, Shirajum; Jesmin, Khaleda; Hruschka, Daniel.

In: Royal Society Open Science, Vol. 6, No. 2, 181386, 01.02.2019.

Research output: Contribution to journalArticle

Tiokhin, Leonid ; Hackman, Joseph ; Munira, Shirajum ; Jesmin, Khaleda ; Hruschka, Daniel. / Generalizability is not optional : Insights from a cross-cultural study of social discounting. In: Royal Society Open Science. 2019 ; Vol. 6, No. 2.
@article{146c99a30929485794b1862dea703bed,
title = "Generalizability is not optional: Insights from a cross-cultural study of social discounting",
abstract = "Current scientific reforms focus more on solutions to the problem of reliability (e.g. direct replications) than generalizability. Here, we use a cross-cultural study of social discounting to illustrate the utility of a complementary focus on generalizability across diverse human populations. Social discounting is the tendency to sacrifice more for socially close individuals-a phenomenon replicated across countries and laboratories. Yet, when adapting a typical protocol to low-literacy, resource-scarce settings in Bangladesh and Indonesia, we find no independent effect of social distance on generosity, despite still documenting this effect among US participants. Several reliability and validity checks suggest that methodological issues alone cannot explain this finding. These results illustrate why we must complement replication efforts with investment in strong checks on generalizability. By failing to do so, we risk developing theories of human nature that reliably explain behaviour among only a thin slice of humanity.",
keywords = "Cross-cultural differences, Generalizability, Replication, Sample diversity, Social discounting",
author = "Leonid Tiokhin and Joseph Hackman and Shirajum Munira and Khaleda Jesmin and Daniel Hruschka",
year = "2019",
month = "2",
day = "1",
doi = "10.1098/rsos.181386",
language = "English (US)",
volume = "6",
journal = "Royal Society Open Science",
issn = "2054-5703",
publisher = "The Royal Society",
number = "2",

}

TY - JOUR

T1 - Generalizability is not optional

T2 - Insights from a cross-cultural study of social discounting

AU - Tiokhin, Leonid

AU - Hackman, Joseph

AU - Munira, Shirajum

AU - Jesmin, Khaleda

AU - Hruschka, Daniel

PY - 2019/2/1

Y1 - 2019/2/1

N2 - Current scientific reforms focus more on solutions to the problem of reliability (e.g. direct replications) than generalizability. Here, we use a cross-cultural study of social discounting to illustrate the utility of a complementary focus on generalizability across diverse human populations. Social discounting is the tendency to sacrifice more for socially close individuals-a phenomenon replicated across countries and laboratories. Yet, when adapting a typical protocol to low-literacy, resource-scarce settings in Bangladesh and Indonesia, we find no independent effect of social distance on generosity, despite still documenting this effect among US participants. Several reliability and validity checks suggest that methodological issues alone cannot explain this finding. These results illustrate why we must complement replication efforts with investment in strong checks on generalizability. By failing to do so, we risk developing theories of human nature that reliably explain behaviour among only a thin slice of humanity.

AB - Current scientific reforms focus more on solutions to the problem of reliability (e.g. direct replications) than generalizability. Here, we use a cross-cultural study of social discounting to illustrate the utility of a complementary focus on generalizability across diverse human populations. Social discounting is the tendency to sacrifice more for socially close individuals-a phenomenon replicated across countries and laboratories. Yet, when adapting a typical protocol to low-literacy, resource-scarce settings in Bangladesh and Indonesia, we find no independent effect of social distance on generosity, despite still documenting this effect among US participants. Several reliability and validity checks suggest that methodological issues alone cannot explain this finding. These results illustrate why we must complement replication efforts with investment in strong checks on generalizability. By failing to do so, we risk developing theories of human nature that reliably explain behaviour among only a thin slice of humanity.

KW - Cross-cultural differences

KW - Generalizability

KW - Replication

KW - Sample diversity

KW - Social discounting

UR - http://www.scopus.com/inward/record.url?scp=85062767097&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85062767097&partnerID=8YFLogxK

U2 - 10.1098/rsos.181386

DO - 10.1098/rsos.181386

M3 - Article

VL - 6

JO - Royal Society Open Science

JF - Royal Society Open Science

SN - 2054-5703

IS - 2

M1 - 181386

ER -