Reliability between online raters with varying familiarities of a region: Microscale Audit of Pedestrian Streetscapes (MAPS)

Wenfei Zhu, Yuliang Sun, Jonathan Kurka, Carrie Geremia, Jessa K. Engelberg, Kelli Cain, Terry Conway, James F. Sallis, Steven P. Hooker, Marc Adams

Research output: Contribution to journalArticle

5 Citations (Scopus)

Abstract

Background To test inter-rater reliability of the online Microscale Audit of Pedestrian Streetscapes (MAPS) tool between raters with varying familiarities of Phoenix, Arizona. Methods The online MAPS tool, based on the MAPS in-field audit tool and scoring system, was used for audits. Sixty route pairs, 141 segment pairs, and 92 crossing pairs in Phoenix were included. Each route, segment or crossing was audited by two independent raters: one rater in Phoenix and the other in San Diego, California, respectively. Item, subscale scores, and total scores reliability analyses were computed using Kappa or intra-class correlation coefficient (ICC). Results The route overall score had substantial reliability (ICC: 0.832). Of the route subscale and overall scores, sixteen out of twenty had moderate to substantial reliability (ICC: 0.616–0.906), and the four subscales had fair reliability (ICC: 0.409–0.563). Sixteen out of twenty scores in segment and crossing sections demonstrated fair to substantial reliability (ICC: 0.448–0.897), and the remaining four had slight reliability (ICC: 0.348–0.364). Conclusions Most of the online MAPS items, subscales, and overall scores demonstrated fair to substantial reliability between raters with varied familiarities of the Phoenix area. Results support use of online MAPS to measure microscale elements of the built environment by raters unfamiliar with a region.

Original languageEnglish (US)
Pages (from-to)240-248
Number of pages9
JournalLandscape and Urban Planning
Volume167
DOIs
StatePublished - Nov 1 2017

Fingerprint

familiarity
pedestrian
audit

Keywords

  • Microscale audit
  • Pedestrian
  • Physical activity
  • Streetscapes
  • walkability

ASJC Scopus subject areas

  • Ecology
  • Nature and Landscape Conservation
  • Management, Monitoring, Policy and Law

Cite this

Reliability between online raters with varying familiarities of a region : Microscale Audit of Pedestrian Streetscapes (MAPS). / Zhu, Wenfei; Sun, Yuliang; Kurka, Jonathan; Geremia, Carrie; Engelberg, Jessa K.; Cain, Kelli; Conway, Terry; Sallis, James F.; Hooker, Steven P.; Adams, Marc.

In: Landscape and Urban Planning, Vol. 167, 01.11.2017, p. 240-248.

Research output: Contribution to journalArticle

Zhu, Wenfei ; Sun, Yuliang ; Kurka, Jonathan ; Geremia, Carrie ; Engelberg, Jessa K. ; Cain, Kelli ; Conway, Terry ; Sallis, James F. ; Hooker, Steven P. ; Adams, Marc. / Reliability between online raters with varying familiarities of a region : Microscale Audit of Pedestrian Streetscapes (MAPS). In: Landscape and Urban Planning. 2017 ; Vol. 167. pp. 240-248.
@article{f6f5d06fc3d04e0187fde9f78b11497e,
title = "Reliability between online raters with varying familiarities of a region: Microscale Audit of Pedestrian Streetscapes (MAPS)",
abstract = "Background To test inter-rater reliability of the online Microscale Audit of Pedestrian Streetscapes (MAPS) tool between raters with varying familiarities of Phoenix, Arizona. Methods The online MAPS tool, based on the MAPS in-field audit tool and scoring system, was used for audits. Sixty route pairs, 141 segment pairs, and 92 crossing pairs in Phoenix were included. Each route, segment or crossing was audited by two independent raters: one rater in Phoenix and the other in San Diego, California, respectively. Item, subscale scores, and total scores reliability analyses were computed using Kappa or intra-class correlation coefficient (ICC). Results The route overall score had substantial reliability (ICC: 0.832). Of the route subscale and overall scores, sixteen out of twenty had moderate to substantial reliability (ICC: 0.616–0.906), and the four subscales had fair reliability (ICC: 0.409–0.563). Sixteen out of twenty scores in segment and crossing sections demonstrated fair to substantial reliability (ICC: 0.448–0.897), and the remaining four had slight reliability (ICC: 0.348–0.364). Conclusions Most of the online MAPS items, subscales, and overall scores demonstrated fair to substantial reliability between raters with varied familiarities of the Phoenix area. Results support use of online MAPS to measure microscale elements of the built environment by raters unfamiliar with a region.",
keywords = "Microscale audit, Pedestrian, Physical activity, Streetscapes, walkability",
author = "Wenfei Zhu and Yuliang Sun and Jonathan Kurka and Carrie Geremia and Engelberg, {Jessa K.} and Kelli Cain and Terry Conway and Sallis, {James F.} and Hooker, {Steven P.} and Marc Adams",
year = "2017",
month = "11",
day = "1",
doi = "10.1016/j.landurbplan.2017.06.014",
language = "English (US)",
volume = "167",
pages = "240--248",
journal = "Landscape and Urban Planning",
issn = "0169-2046",
publisher = "Elsevier",

}

TY - JOUR

T1 - Reliability between online raters with varying familiarities of a region

T2 - Microscale Audit of Pedestrian Streetscapes (MAPS)

AU - Zhu, Wenfei

AU - Sun, Yuliang

AU - Kurka, Jonathan

AU - Geremia, Carrie

AU - Engelberg, Jessa K.

AU - Cain, Kelli

AU - Conway, Terry

AU - Sallis, James F.

AU - Hooker, Steven P.

AU - Adams, Marc

PY - 2017/11/1

Y1 - 2017/11/1

N2 - Background To test inter-rater reliability of the online Microscale Audit of Pedestrian Streetscapes (MAPS) tool between raters with varying familiarities of Phoenix, Arizona. Methods The online MAPS tool, based on the MAPS in-field audit tool and scoring system, was used for audits. Sixty route pairs, 141 segment pairs, and 92 crossing pairs in Phoenix were included. Each route, segment or crossing was audited by two independent raters: one rater in Phoenix and the other in San Diego, California, respectively. Item, subscale scores, and total scores reliability analyses were computed using Kappa or intra-class correlation coefficient (ICC). Results The route overall score had substantial reliability (ICC: 0.832). Of the route subscale and overall scores, sixteen out of twenty had moderate to substantial reliability (ICC: 0.616–0.906), and the four subscales had fair reliability (ICC: 0.409–0.563). Sixteen out of twenty scores in segment and crossing sections demonstrated fair to substantial reliability (ICC: 0.448–0.897), and the remaining four had slight reliability (ICC: 0.348–0.364). Conclusions Most of the online MAPS items, subscales, and overall scores demonstrated fair to substantial reliability between raters with varied familiarities of the Phoenix area. Results support use of online MAPS to measure microscale elements of the built environment by raters unfamiliar with a region.

AB - Background To test inter-rater reliability of the online Microscale Audit of Pedestrian Streetscapes (MAPS) tool between raters with varying familiarities of Phoenix, Arizona. Methods The online MAPS tool, based on the MAPS in-field audit tool and scoring system, was used for audits. Sixty route pairs, 141 segment pairs, and 92 crossing pairs in Phoenix were included. Each route, segment or crossing was audited by two independent raters: one rater in Phoenix and the other in San Diego, California, respectively. Item, subscale scores, and total scores reliability analyses were computed using Kappa or intra-class correlation coefficient (ICC). Results The route overall score had substantial reliability (ICC: 0.832). Of the route subscale and overall scores, sixteen out of twenty had moderate to substantial reliability (ICC: 0.616–0.906), and the four subscales had fair reliability (ICC: 0.409–0.563). Sixteen out of twenty scores in segment and crossing sections demonstrated fair to substantial reliability (ICC: 0.448–0.897), and the remaining four had slight reliability (ICC: 0.348–0.364). Conclusions Most of the online MAPS items, subscales, and overall scores demonstrated fair to substantial reliability between raters with varied familiarities of the Phoenix area. Results support use of online MAPS to measure microscale elements of the built environment by raters unfamiliar with a region.

KW - Microscale audit

KW - Pedestrian

KW - Physical activity

KW - Streetscapes

KW - walkability

UR - http://www.scopus.com/inward/record.url?scp=85026847663&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85026847663&partnerID=8YFLogxK

U2 - 10.1016/j.landurbplan.2017.06.014

DO - 10.1016/j.landurbplan.2017.06.014

M3 - Article

AN - SCOPUS:85026847663

VL - 167

SP - 240

EP - 248

JO - Landscape and Urban Planning

JF - Landscape and Urban Planning

SN - 0169-2046

ER -