Effects of expectancy on assessing covariation in data: "Prior belief" versus "meaning"

A large number of researchers have addressed the question of how prior beliefs affect assessment of covariation in new data. Some have suggested that prior beliefs disrupt covariation assessment (Nisbett & Ross, 1980), while others have claimed they help (Wright & Murphy, 1984). Research in this tradition has not consistently distinguished meaningfulness of the data from expectations about the particular relationship between the variables to be assessed. We collected covariance judgments on meaningful variable pairs where subjects had a prior belief in a positive relation, had a prior belief in a negative relation, had a prior belief that the variables are unrelated, or were agnostic about the existence or nature of relation. Subjects rated data with negative, positive, and zero correlations. We evaluated performance in terms of subjects' ability to discriminate objectively different correlations, rather than simply comparing to a reference statistic, and also on the bias subjects showed. Subjects with no prior belief, with positive beliefs, and with negative beliefs were all reasonably well able to discriminate among different objective correlations. In addition, subjects with no prior belief showed appropriate use of the judgment scale, while those having a positive or negative expectation were biased in the direction of their prior belief. In contrast, subjects with the prior belief that the variables were unrelated showed essentially no discrimination. Our results disconfirm the hypothesis that prior beliefs generally facilitate correlation assessment of summarized data. Judgments of meaningful data were best when subjects were initially agnostic.