The relationship between National Resident Match Program rank and perceived performance in an emergency medicine residency

A nonconcurrent prospective cohort study was conducted to evaluate if National Resident Matching Program (NRMP) rank developed using multivariate regression followed by consensus group activity is associated with perceived general performance during emergency medicine residency. All residents graduating from a university hospital-based residency program between 1990 to 1993 were ranked by university faculty, private attendings, charge nurses, end a clerk. Each evaluator was asked to order (from strongest to weakest) a deck of cards that contained only each graduate's name and picture. NRMP ranks were scaled to adjust for differences in each year's match list length. Evaluators were unaware of the graduates' NRMP ranks. Agreement among raters was analyzed using Kendall's coefficient of concordance. Perceived ranks were compared with actual NRMP ranks using the Spearman correlation procedure. Twenty graduates were evaluated by eight different individuals, yielding 160 pairs of ranks. There was moderately strong agreement among evaluators about the relative strength of the 20 residents (W = 0.07, P < .001). The association of perceived rank with NRMP rank was much greater than that expected by chance (r(s) = .35, P < .0001). Applicants with better NRMP ranks were perceived as stronger residents, which supports the strategy of using formal statistical modelling followed by consensus group activity as a method of generating NRMP rank lists.