Benchmarking veterans affairs medical centers in the delivery of preventive health services: Comparison of methods

Bradley Doebbeling, Thomas E. Vaughn, Robert F. Woolson, Paul M. Peloso, Marcia M. Ward, Elena Letuchy, Bonnie J. BootsMiller, Toni Tripp-Reimer, Laurence G. Branch

Research output: Contribution to journalArticle

31 Citations (Scopus)

Abstract

OBJECTIVE. To identify consistent provision of clinical preventive services, we sought to benchmark all acute care Veterans Affairs Medical Centers (VAMCs) against each other nationally on the basis of multiple evidence-based, performance measures to identify facilities performing consistently higher and lower than expected. METHODS. The 1998 Veterans Health Survey assessed the self-reported delivery of evidence-based clinical preventive services in a stratified national sample of 450 ambulatory care patients seen at each VAMC. Proportions appropriately receiving each service within the recommended time interval were calculated for 138 VAMCs. Percentile ranks for each outcome were assigned. Two approaches were used for benchmarking performance. First, a scaled score for each facility was calculated across the set of 12 measures. Second, facilities were ranked based on the sum of the percentile ranks over a range of specific high cutoffs (eg, 70-80%) and above a range of lower cutoffs (eg, 40-50%). Ranking was validated by comparing with deciles of ranks on chart audit (External Peer Review Program, EPRP) data using Kendall's τ-b and χ2 quality-of-fit test. Differences between consistently high adherence (CHA) and low adherence (CLA) facilities were compared using the Wilcoxon rank sum test on 14 VHS and 11 EPRP outcomes. RESULTS. Data from 39,939 patients (67% response rate) were examined. In combination, cutoffs of greater than 50th percentile and greater than 75th percentile rank yielded 12 of 14 VHS and 6 of 11 EPRP measures different between CHA and CLA facilities. The scaled-score approach resulted in 20 CHA and 14 CLA facilities. The sum of outcomes ranked above 50th percentile and over 75th percentile for CHA facilities (n = 17) was 15 or more. The sum of outcomes ranked above the same cutoffs for CLA facilities (n = 16) was 3 or less. EPRP and 1998 VHS data demonstrated that the survey measures and benchmarking approaches were both reliable and valid. Both approaches resulted in multiple differences between CHA and CLA facilities; differences were greater using the percentile rank approach. CONCLUSIONS. The VA has successfully encouraged adoption of evidence-based clinical preventive services throughout its health care system. However, facilities show wide variation in their levels of delivery and can be distinguished on the basis of their consistently high or low levels of adherence. Examining service delivery across multiple performance indicators allows identification of opportunities to improve clinical practice guideline implementation and the delivery of preventive services. This approach identifies model institutions where focused investigation of factors associated with consistent performance may be particularly fruitful.

Original languageEnglish (US)
Pages (from-to)540-554
Number of pages15
JournalMedical Care
Volume40
Issue number6
DOIs
StatePublished - Jun 2002
Externally publishedYes

Fingerprint

Preventive Health Services
comparison of methods
Benchmarking
Peer Review
benchmarking
Veterans
health service
Nonparametric Statistics
peer review
Veterans Health
Ambulatory Care
Health Surveys
Practice Guidelines
performance
Delivery of Health Care
evidence
audit
patient care
ranking

Keywords

  • Benchmarking
  • Clinical performance
  • Clinical preventive services
  • Health services research methodology
  • Quality of care

ASJC Scopus subject areas

  • Public Health, Environmental and Occupational Health
  • Nursing(all)
  • Health(social science)
  • Health Professions(all)

Cite this

Benchmarking veterans affairs medical centers in the delivery of preventive health services : Comparison of methods. / Doebbeling, Bradley; Vaughn, Thomas E.; Woolson, Robert F.; Peloso, Paul M.; Ward, Marcia M.; Letuchy, Elena; BootsMiller, Bonnie J.; Tripp-Reimer, Toni; Branch, Laurence G.

In: Medical Care, Vol. 40, No. 6, 06.2002, p. 540-554.

Research output: Contribution to journalArticle

Doebbeling, B, Vaughn, TE, Woolson, RF, Peloso, PM, Ward, MM, Letuchy, E, BootsMiller, BJ, Tripp-Reimer, T & Branch, LG 2002, 'Benchmarking veterans affairs medical centers in the delivery of preventive health services: Comparison of methods', Medical Care, vol. 40, no. 6, pp. 540-554. https://doi.org/10.1097/00005650-200206000-00011
Doebbeling, Bradley ; Vaughn, Thomas E. ; Woolson, Robert F. ; Peloso, Paul M. ; Ward, Marcia M. ; Letuchy, Elena ; BootsMiller, Bonnie J. ; Tripp-Reimer, Toni ; Branch, Laurence G. / Benchmarking veterans affairs medical centers in the delivery of preventive health services : Comparison of methods. In: Medical Care. 2002 ; Vol. 40, No. 6. pp. 540-554.
@article{e030686703a441ed8090b5f5e2731b2d,
title = "Benchmarking veterans affairs medical centers in the delivery of preventive health services: Comparison of methods",
abstract = "OBJECTIVE. To identify consistent provision of clinical preventive services, we sought to benchmark all acute care Veterans Affairs Medical Centers (VAMCs) against each other nationally on the basis of multiple evidence-based, performance measures to identify facilities performing consistently higher and lower than expected. METHODS. The 1998 Veterans Health Survey assessed the self-reported delivery of evidence-based clinical preventive services in a stratified national sample of 450 ambulatory care patients seen at each VAMC. Proportions appropriately receiving each service within the recommended time interval were calculated for 138 VAMCs. Percentile ranks for each outcome were assigned. Two approaches were used for benchmarking performance. First, a scaled score for each facility was calculated across the set of 12 measures. Second, facilities were ranked based on the sum of the percentile ranks over a range of specific high cutoffs (eg, 70-80{\%}) and above a range of lower cutoffs (eg, 40-50{\%}). Ranking was validated by comparing with deciles of ranks on chart audit (External Peer Review Program, EPRP) data using Kendall's τ-b and χ2 quality-of-fit test. Differences between consistently high adherence (CHA) and low adherence (CLA) facilities were compared using the Wilcoxon rank sum test on 14 VHS and 11 EPRP outcomes. RESULTS. Data from 39,939 patients (67{\%} response rate) were examined. In combination, cutoffs of greater than 50th percentile and greater than 75th percentile rank yielded 12 of 14 VHS and 6 of 11 EPRP measures different between CHA and CLA facilities. The scaled-score approach resulted in 20 CHA and 14 CLA facilities. The sum of outcomes ranked above 50th percentile and over 75th percentile for CHA facilities (n = 17) was 15 or more. The sum of outcomes ranked above the same cutoffs for CLA facilities (n = 16) was 3 or less. EPRP and 1998 VHS data demonstrated that the survey measures and benchmarking approaches were both reliable and valid. Both approaches resulted in multiple differences between CHA and CLA facilities; differences were greater using the percentile rank approach. CONCLUSIONS. The VA has successfully encouraged adoption of evidence-based clinical preventive services throughout its health care system. However, facilities show wide variation in their levels of delivery and can be distinguished on the basis of their consistently high or low levels of adherence. Examining service delivery across multiple performance indicators allows identification of opportunities to improve clinical practice guideline implementation and the delivery of preventive services. This approach identifies model institutions where focused investigation of factors associated with consistent performance may be particularly fruitful.",
keywords = "Benchmarking, Clinical performance, Clinical preventive services, Health services research methodology, Quality of care",
author = "Bradley Doebbeling and Vaughn, {Thomas E.} and Woolson, {Robert F.} and Peloso, {Paul M.} and Ward, {Marcia M.} and Elena Letuchy and BootsMiller, {Bonnie J.} and Toni Tripp-Reimer and Branch, {Laurence G.}",
year = "2002",
month = "6",
doi = "10.1097/00005650-200206000-00011",
language = "English (US)",
volume = "40",
pages = "540--554",
journal = "Medical Care",
issn = "0025-7079",
publisher = "Lippincott Williams and Wilkins",
number = "6",

}

TY - JOUR

T1 - Benchmarking veterans affairs medical centers in the delivery of preventive health services

T2 - Comparison of methods

AU - Doebbeling, Bradley

AU - Vaughn, Thomas E.

AU - Woolson, Robert F.

AU - Peloso, Paul M.

AU - Ward, Marcia M.

AU - Letuchy, Elena

AU - BootsMiller, Bonnie J.

AU - Tripp-Reimer, Toni

AU - Branch, Laurence G.

PY - 2002/6

Y1 - 2002/6

N2 - OBJECTIVE. To identify consistent provision of clinical preventive services, we sought to benchmark all acute care Veterans Affairs Medical Centers (VAMCs) against each other nationally on the basis of multiple evidence-based, performance measures to identify facilities performing consistently higher and lower than expected. METHODS. The 1998 Veterans Health Survey assessed the self-reported delivery of evidence-based clinical preventive services in a stratified national sample of 450 ambulatory care patients seen at each VAMC. Proportions appropriately receiving each service within the recommended time interval were calculated for 138 VAMCs. Percentile ranks for each outcome were assigned. Two approaches were used for benchmarking performance. First, a scaled score for each facility was calculated across the set of 12 measures. Second, facilities were ranked based on the sum of the percentile ranks over a range of specific high cutoffs (eg, 70-80%) and above a range of lower cutoffs (eg, 40-50%). Ranking was validated by comparing with deciles of ranks on chart audit (External Peer Review Program, EPRP) data using Kendall's τ-b and χ2 quality-of-fit test. Differences between consistently high adherence (CHA) and low adherence (CLA) facilities were compared using the Wilcoxon rank sum test on 14 VHS and 11 EPRP outcomes. RESULTS. Data from 39,939 patients (67% response rate) were examined. In combination, cutoffs of greater than 50th percentile and greater than 75th percentile rank yielded 12 of 14 VHS and 6 of 11 EPRP measures different between CHA and CLA facilities. The scaled-score approach resulted in 20 CHA and 14 CLA facilities. The sum of outcomes ranked above 50th percentile and over 75th percentile for CHA facilities (n = 17) was 15 or more. The sum of outcomes ranked above the same cutoffs for CLA facilities (n = 16) was 3 or less. EPRP and 1998 VHS data demonstrated that the survey measures and benchmarking approaches were both reliable and valid. Both approaches resulted in multiple differences between CHA and CLA facilities; differences were greater using the percentile rank approach. CONCLUSIONS. The VA has successfully encouraged adoption of evidence-based clinical preventive services throughout its health care system. However, facilities show wide variation in their levels of delivery and can be distinguished on the basis of their consistently high or low levels of adherence. Examining service delivery across multiple performance indicators allows identification of opportunities to improve clinical practice guideline implementation and the delivery of preventive services. This approach identifies model institutions where focused investigation of factors associated with consistent performance may be particularly fruitful.

AB - OBJECTIVE. To identify consistent provision of clinical preventive services, we sought to benchmark all acute care Veterans Affairs Medical Centers (VAMCs) against each other nationally on the basis of multiple evidence-based, performance measures to identify facilities performing consistently higher and lower than expected. METHODS. The 1998 Veterans Health Survey assessed the self-reported delivery of evidence-based clinical preventive services in a stratified national sample of 450 ambulatory care patients seen at each VAMC. Proportions appropriately receiving each service within the recommended time interval were calculated for 138 VAMCs. Percentile ranks for each outcome were assigned. Two approaches were used for benchmarking performance. First, a scaled score for each facility was calculated across the set of 12 measures. Second, facilities were ranked based on the sum of the percentile ranks over a range of specific high cutoffs (eg, 70-80%) and above a range of lower cutoffs (eg, 40-50%). Ranking was validated by comparing with deciles of ranks on chart audit (External Peer Review Program, EPRP) data using Kendall's τ-b and χ2 quality-of-fit test. Differences between consistently high adherence (CHA) and low adherence (CLA) facilities were compared using the Wilcoxon rank sum test on 14 VHS and 11 EPRP outcomes. RESULTS. Data from 39,939 patients (67% response rate) were examined. In combination, cutoffs of greater than 50th percentile and greater than 75th percentile rank yielded 12 of 14 VHS and 6 of 11 EPRP measures different between CHA and CLA facilities. The scaled-score approach resulted in 20 CHA and 14 CLA facilities. The sum of outcomes ranked above 50th percentile and over 75th percentile for CHA facilities (n = 17) was 15 or more. The sum of outcomes ranked above the same cutoffs for CLA facilities (n = 16) was 3 or less. EPRP and 1998 VHS data demonstrated that the survey measures and benchmarking approaches were both reliable and valid. Both approaches resulted in multiple differences between CHA and CLA facilities; differences were greater using the percentile rank approach. CONCLUSIONS. The VA has successfully encouraged adoption of evidence-based clinical preventive services throughout its health care system. However, facilities show wide variation in their levels of delivery and can be distinguished on the basis of their consistently high or low levels of adherence. Examining service delivery across multiple performance indicators allows identification of opportunities to improve clinical practice guideline implementation and the delivery of preventive services. This approach identifies model institutions where focused investigation of factors associated with consistent performance may be particularly fruitful.

KW - Benchmarking

KW - Clinical performance

KW - Clinical preventive services

KW - Health services research methodology

KW - Quality of care

UR - http://www.scopus.com/inward/record.url?scp=0036615989&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0036615989&partnerID=8YFLogxK

U2 - 10.1097/00005650-200206000-00011

DO - 10.1097/00005650-200206000-00011

M3 - Article

C2 - 12021680

AN - SCOPUS:0036615989

VL - 40

SP - 540

EP - 554

JO - Medical Care

JF - Medical Care

SN - 0025-7079

IS - 6

ER -