Judging the judges

The evaluation of judges, especially circuit court judges, has commanded increased attention, with the quantitative analyses of Stephen Choi and Mitu Gulati. However, the proper dimensions for the evaluation of judges remains much disputed. Critics have challenged Choi & Gulati's scales for measuring judicial quality but have offered little that is positive that would improve measurement. The critics make philosophical challenges to whether the measures truly capture the qualities of judging we should desire, but they offer no measurement tools to improve on Choi and Gulati. We hope to advance the theoretical and empirical evaluation by incorporating different scales for evaluating judges. We consider Choi & Gulati's data, plus the record of each judge's opinions on review at the Supreme Court and other more subjective measures of judicial quality. We then employ a cluster analysis to differentiate among different types of judges on all these dimensions. This analysis does not provide a rating of the "best" judges, because the standards for "best" judging are contested. Some judges may be considered best, because their opinions receive more citations. However, these very judges may be considered less than best, because they render unnecessarily expansive decisions that yield more citations. Some argue for judges with minimalist decisionmaking characteristics that tend to result in fewer citations. Our cluster analysis simply categorizes circuit court judges into groups with like decisionmaking characteristics. This enables an analysis of which group, or type of judge, should be considered "best."