Statistics for program assessment: Has the program made a difference?

As funding becomes scarcer and the demand for accountability increases, creditable assessment and evaluation become more important. For example, funding is generally scarce for programs to establish and to improve activities designed to increase enrollment and retention in engineering. Therefore, almost all funding allocated to these recruitment and retention activities requires an assessment of the program to see if the money and time have been well spent. This paper describes basic statistical concepts that should be considered when assessing a program or activity. Examples are given to illustrate both good and poor program assessment. Warnings are given for data that may turn out to be useless and suggestions presented on ways to enhance data presentation. What it takes for data to be "significant" will also be discussed, as well as the problem of sample size. Without the proper planning of assessments and data collection, it may be very difficult to show that the program has made a difference. If a program director does not have a good statistical background, they would be well advised to have an assessment person on their team to help plan assessment strategy, to analyze the data, and to draw conclusions.