Efficacy of peer review network structures: The effects of reciprocity and clustering

Scott Stevens, Andrew Waters, Dmytro Babik, David Tinapple

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Scopus citations

Abstract

IT-enabled peer-based creation, review, and evaluation systems are widely spread in multiple areas of open innovation and knowledge management. Despite a noticeable variety of designs, particularly, in the structure of peer networks (that is, how participants are linked to each other as creators and reviewers), these design choices are hardly ever grounded in design research. Characteristics of peer network structure, such as reciprocity and clustering, may affect how well such systems reveal participants' competencies and their products' qualities. Designing peer review systems that produce valid and reliable evaluations is, therefore, among the most fundamental concerns. Using a simulation approach, we show that reciprocity and clustering indeed have an effect, but its direction and magnitude depend on the evaluation scale used. So far, we have found no evidence that transitional networks have superior efficacy in comparison with "pure" networks. We outlined directions for further investigation.

Original languageEnglish (US)
Title of host publication2016 International Conference on Information Systems, ICIS 2016
PublisherAssociation for Information Systems
ISBN (Electronic)9780996683135
StatePublished - 2016
Event2016 International Conference on Information Systems, ICIS 2016 - Dublin, Ireland
Duration: Dec 11 2016Dec 14 2016

Other

Other2016 International Conference on Information Systems, ICIS 2016
CountryIreland
CityDublin
Period12/11/1612/14/16

Keywords

  • Clustering
  • Evaluation systems
  • Information quality
  • Knowledge artifacts
  • Network structures
  • Peer review
  • Reciprocity

ASJC Scopus subject areas

  • Information Systems

Fingerprint Dive into the research topics of 'Efficacy of peer review network structures: The effects of reciprocity and clustering'. Together they form a unique fingerprint.

  • Cite this

    Stevens, S., Waters, A., Babik, D., & Tinapple, D. (2016). Efficacy of peer review network structures: The effects of reciprocity and clustering. In 2016 International Conference on Information Systems, ICIS 2016 Association for Information Systems.