Generalizability is not optional: Insights from a cross-cultural study of social discounting

Leonid Tiokhin, Joseph Hackman, Shirajum Munira, Khaleda Jesmin, Daniel Hruschka

    Research output: Contribution to journalArticle

    Abstract

    Current scientific reforms focus more on solutions to the problem of reliability (e.g. direct replications) than generalizability. Here, we use a cross-cultural study of social discounting to illustrate the utility of a complementary focus on generalizability across diverse human populations. Social discounting is the tendency to sacrifice more for socially close individuals-a phenomenon replicated across countries and laboratories. Yet, when adapting a typical protocol to low-literacy, resource-scarce settings in Bangladesh and Indonesia, we find no independent effect of social distance on generosity, despite still documenting this effect among US participants. Several reliability and validity checks suggest that methodological issues alone cannot explain this finding. These results illustrate why we must complement replication efforts with investment in strong checks on generalizability. By failing to do so, we risk developing theories of human nature that reliably explain behaviour among only a thin slice of humanity.

    Original languageEnglish (US)
    Article number181386
    JournalRoyal Society Open Science
    Volume6
    Issue number2
    DOIs
    StatePublished - Feb 1 2019

    Fingerprint

    cultural studies
    social distance
    Bangladesh
    Indonesia
    literacy
    reform
    resources

    Keywords

    • Cross-cultural differences
    • Generalizability
    • Replication
    • Sample diversity
    • Social discounting

    ASJC Scopus subject areas

    • General

    Cite this

    Generalizability is not optional : Insights from a cross-cultural study of social discounting. / Tiokhin, Leonid; Hackman, Joseph; Munira, Shirajum; Jesmin, Khaleda; Hruschka, Daniel.

    In: Royal Society Open Science, Vol. 6, No. 2, 181386, 01.02.2019.

    Research output: Contribution to journalArticle

    Tiokhin, Leonid ; Hackman, Joseph ; Munira, Shirajum ; Jesmin, Khaleda ; Hruschka, Daniel. / Generalizability is not optional : Insights from a cross-cultural study of social discounting. In: Royal Society Open Science. 2019 ; Vol. 6, No. 2.
    @article{146c99a30929485794b1862dea703bed,
    title = "Generalizability is not optional: Insights from a cross-cultural study of social discounting",
    abstract = "Current scientific reforms focus more on solutions to the problem of reliability (e.g. direct replications) than generalizability. Here, we use a cross-cultural study of social discounting to illustrate the utility of a complementary focus on generalizability across diverse human populations. Social discounting is the tendency to sacrifice more for socially close individuals-a phenomenon replicated across countries and laboratories. Yet, when adapting a typical protocol to low-literacy, resource-scarce settings in Bangladesh and Indonesia, we find no independent effect of social distance on generosity, despite still documenting this effect among US participants. Several reliability and validity checks suggest that methodological issues alone cannot explain this finding. These results illustrate why we must complement replication efforts with investment in strong checks on generalizability. By failing to do so, we risk developing theories of human nature that reliably explain behaviour among only a thin slice of humanity.",
    keywords = "Cross-cultural differences, Generalizability, Replication, Sample diversity, Social discounting",
    author = "Leonid Tiokhin and Joseph Hackman and Shirajum Munira and Khaleda Jesmin and Daniel Hruschka",
    year = "2019",
    month = "2",
    day = "1",
    doi = "10.1098/rsos.181386",
    language = "English (US)",
    volume = "6",
    journal = "Royal Society Open Science",
    issn = "2054-5703",
    publisher = "The Royal Society",
    number = "2",

    }

    TY - JOUR

    T1 - Generalizability is not optional

    T2 - Insights from a cross-cultural study of social discounting

    AU - Tiokhin, Leonid

    AU - Hackman, Joseph

    AU - Munira, Shirajum

    AU - Jesmin, Khaleda

    AU - Hruschka, Daniel

    PY - 2019/2/1

    Y1 - 2019/2/1

    N2 - Current scientific reforms focus more on solutions to the problem of reliability (e.g. direct replications) than generalizability. Here, we use a cross-cultural study of social discounting to illustrate the utility of a complementary focus on generalizability across diverse human populations. Social discounting is the tendency to sacrifice more for socially close individuals-a phenomenon replicated across countries and laboratories. Yet, when adapting a typical protocol to low-literacy, resource-scarce settings in Bangladesh and Indonesia, we find no independent effect of social distance on generosity, despite still documenting this effect among US participants. Several reliability and validity checks suggest that methodological issues alone cannot explain this finding. These results illustrate why we must complement replication efforts with investment in strong checks on generalizability. By failing to do so, we risk developing theories of human nature that reliably explain behaviour among only a thin slice of humanity.

    AB - Current scientific reforms focus more on solutions to the problem of reliability (e.g. direct replications) than generalizability. Here, we use a cross-cultural study of social discounting to illustrate the utility of a complementary focus on generalizability across diverse human populations. Social discounting is the tendency to sacrifice more for socially close individuals-a phenomenon replicated across countries and laboratories. Yet, when adapting a typical protocol to low-literacy, resource-scarce settings in Bangladesh and Indonesia, we find no independent effect of social distance on generosity, despite still documenting this effect among US participants. Several reliability and validity checks suggest that methodological issues alone cannot explain this finding. These results illustrate why we must complement replication efforts with investment in strong checks on generalizability. By failing to do so, we risk developing theories of human nature that reliably explain behaviour among only a thin slice of humanity.

    KW - Cross-cultural differences

    KW - Generalizability

    KW - Replication

    KW - Sample diversity

    KW - Social discounting

    UR - http://www.scopus.com/inward/record.url?scp=85062767097&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=85062767097&partnerID=8YFLogxK

    U2 - 10.1098/rsos.181386

    DO - 10.1098/rsos.181386

    M3 - Article

    VL - 6

    JO - Royal Society Open Science

    JF - Royal Society Open Science

    SN - 2054-5703

    IS - 2

    M1 - 181386

    ER -