Guidelines for samplers: Measuring a change in behaviour from before and after surveys

This paper addresses the issue of using before and after surveys to evaluate behavioural changes in response to transport policies and investments, a procedure that, we argue is done far too rarely in this profession. Further, it demonstrates very clearly that, in almost all conceivable cases, there are considerable economies to be obtained by using a panel (again, under-utilised in our profession) to undertake evaluation, rather than successive independent cross-sectional surveys. The paper also addresses the critical issue of sample size requirements for measuring changes of a relatively small magnitude in travel behaviour; i.e., to state, with 95% confidence, that if there is a ρ percent change in behaviour for the sample, there is a ρ percent ± e percent change in the behaviour of the population, where e is the sampling error. In this paper, we present a method for calculating such sample size requirements from first principles and demonstrate the applicability both hypothetically and then empirically using data from the Puget Sound Transportation Panel. The formulation enables designers of before and after surveys to investigate the trade-offs between the statistical accuracy of their predictions and the sample size requirements systematically, without the need to specify ρ a priori. This latter point is crucial, we argue, because we have limited information on ρ, yet, as we explain here, it drives the sample size requirements using alternative, well-cited approaches for calculating sample sizes to assess behavioural change. The results have important ramifications both for those implementing transport policies intended to produce behavioural change, especially when a cost-benefit evaluation of the policy is desired, and for those interpreting the results reported in previous studies.