Measuring the success of virtual tutoring

Starting spring 1996, the Del E. Webb School of Construction (DEWSC) at Arizona State University (ASU) implemented a new system of teaching engineering courses. The lectures became more like social events, with issues being discussed rather than merely presented. Performance measurements that were developed and performed, as well as the comments of the students, indicate that the system has been very successful. The system combines the advantages of distance education (mainly WWW and email communication) with the use of classical textbooks and cooperative learning to enhance the preparation of students BETWEEN lectures. The students can also use the services of a `virtual tutor.' The virtual tutor offers quick-response help through email. This paper describes the adopted method and focuses on the measurements that were developed to assess the success and effectiveness of the system. The main hypothesis of the paper is that the academic performance of a class (i.e. the average grade of the class) can be predicted based on the previous performance of each student. The performance prediction, called `expected performance', takes into account the general performance of the students, the performance in pre-requisite classes, and the grading characteristics of each class. Because the actual performance of the class is influenced by the way in which the class is taught, a comparison between the expected grade and the achieved grade can serve as a basis for measuring the effectiveness of teaching. This paper presents the results for the classes taught with the system described above as well as the measured advantages of using the services of the virtual tutor. It also explains how the developed measurements can be used to select the composition of student teams in cooperative learning.