This study examined what types of passage and individual item attributes of reading ability tests affect item difficulty. Ninety-six questions from the Gates-MacGinitie Reading Test for 7/9th graders were analyzed in terms of individual item characteristics and passage features that were considered to affect the item's difficulty. Whereas two passage features, word frequency and sentence length, influenced the item difficulty, none of the individual question characteristics were found to affect the item difficulty. Thus, the GMRT may be assessing how difficult a passage readers can comprehend, not how difficult a question readers can answer about a passage. These findings imply that the GMRT may function well when identifying basic reading comprehension skill differences, but not higher level reading skill differences. However, these findings may be the result of the passages in the GMRT having an atypical construction. Specifically, shorter passages tended to be composed of longer sentences.