Assessing faculty and organizational change in a professional development program with workshops and disciplinary communities of practice

To achieve more effective teaching and learning in undergraduate engineering education, professional development needs to promote change toward innovation of use of evidence-based instructional strategies (EBIS) in faculty beliefs and practice as well as change in organizational attitudes and norms. To develop an effective program, a strategy for change needs to be utilized and assessed at the individual level and the organizational level. Rogers Diffusion of Innovation (DOI) is an individual change model which consists of five steps of a progression towards any type of adoption of innovation change such as sports, business or teaching. The steps for adoption of change consist of awareness, interest, evaluation, trial, and adoption. Coburn's organizational innovation (COI) change model for organizations consists of four components necessary to support broader change in an educational organization, which include the following. Depth is deep change in faculty beliefs and practice. Sustainability of Innovation refers to flexibility of innovation in differing contexts or disciplines. Spread of Innovation refers to change in beliefs, norms, and principles of individuals across an organization. Shift of Ownership refers to shift in ownership from external facilitators to internal ownership by individuals and their organization. These change theories have been used to develop and assess a large scale NSF IUSE project at a large southwestern university called Just-in-Time-Teaching with Two Way Formative Feedback for Multiple Disciplines (JTFD). The first year of the project has trained 43 faculty in four disciplines using a train-the-trainer model with one semester of 8 biweekly workshops followed by a second semester of 6 disciplinary community of practice (CoP) discussion sessions. Assessment of elements of the two change theories was done with pre-post measurements of participants with surveys, questions and classroom observations. For the DOI model, Awareness of evidence-based instructional strategies (EBIS) increased an average of 31% for topics such as instructional design and active learning. For DOI Interest and Evaluation a survey was created using expectancy-value theory called Value, Expectancy, and Cost of Testing Educational Reforms Survey (VECTERS). For three EBIS strategies of real-world applications, student-to-student discussions, and formative feedback there were 4% to 12% gains for motivation (expectancy, value, and lowered cost). For DOI Trial 91% of faculty agreed or strongly agreed that "the topics discussed provided me with new ideas for implementation and/or reaffirmed strategies I am currently implementing." For DOI Adoption, a classroom observation tool, RTOP, (Reformed Teaching Observational Protocol) showed a gain of 22% of classroom practice toward EBIS. For the COI model, Depth of Faculty Beliefs there was an average gain of 26% in faculty who said that they were sometimes or frequently using EBIS strategies of active learning, cooperative learning, objectives and Blooms's taxonomy. For Sustainability 96% of faculty agreed or strongly agreed that the JTFD project has been successful in creating CoPs which support innovation, implementation, and open dialogue between colleagues." For COI Spread of Innovation, 91% of faculty agreed or strongly agreed that "the topics discussed provided me with new ideas for implementation and/or reaffirmed strategies I am currently implementing." For Shift of Ownership 100% of faculty agreed or strongly agreed that "the tools, strategies, and interactions in the JTFD project would be of value to their future instructional practice and career success." Overall, assessment using the DOI and COI change models demonstrate faculty change and organizational change toward innovation through use of EBIS in teaching and learning.