Towards ecological validity in menu research

The goal of this research was to test the effects of menu organization on user performance under situations representative of typical user-computer interactions. Effects of alphabetical, categorical, and random organizations on response time and accuracy were tested using a problem-solving task in which problems differed in degree of complexity. In addition, explicit or implicit targets were searched for in menus consisting of items from distinct or overlapping categories. Results indicated that alphabetical and categorical organizations were generally equivalent and superior to random organizations, replicating the results of others. Unexpectedly, decreasing category distinctiveness, although generally detrimental, did not seem to negatively affect performance with categorical menu organizations. On the other hand, problem complexity had large effects on performance and magnified the effects of other factors such as target type. These results extend some previous conclusions about menu organization to situations that are more typical of user interactions. The limits of generalizing from results of controlled experiments to truly ecologically valid settings are also discussed.