Deciding how much data you need to collect

The target number of interviews will depend on the number of users and how accurate you would like your data to be and can be expressed in terms of confidence intervals and probability. It will also be partly dependent on the resources you have available.

The statistical aspect of all evaluations can cause a headache for many people but we have included some technical information to ensure that it is clear what you are actually measuring. There is no single or definitive answer to the question ‘what is the economic benefit of the path network?’ because you cannot interview every user. The statistics presented are designed to give you an idea of the accuracy of the responses and how confident you can be that they reflect the ‘real’ situation. We recognise that this is complex, but information about the level of confidence is a vital part of any report and should always be included - it will give your evaluation more credibility in statistical terms to provide the reliability of your conclusions.

Who to Interview

Because you are only interviewing a sample of all users, you need to decide whom to interview without introducing any bias to the evaluation. The simplest method is to invite the next person who passes, and if this is a group, to select one respondent as the person whose birthday is next.

Interview numbers

The actual number of interviews (sample size) required can be statistically related to the degree of accuracy you are seeking. There is a balance between having enough responses to be confident that the results are reliable and interviewing more people than necessary (although the latter is rarely a problem). The final decision may also be influenced by the resources you have available to collect responses, but having a good idea of how to make your results more accurate may help you to plan what resources you need.

You will probably want to be at least 95% certain that, if you find that 50% of the path users are visitors, there really are somewhere between 45% and 55% visitors who use the network (i.e. a 10% range or a 5% confidence interval). To attain this level of confidence in your data you would need to interview 384 people. If you want to be sure that visitors actually make up between 47.5% and 52.5% of the total users (the narrower range is a smaller confidence interval of 2.5%), you would need to interview 1,537 people. The reason that this is important is highlighted in section 5.1.

About the toolkit

This toolkit has been designed and written by Walking-the-Talk based on research and initial development by the Sport Industry Research Centre at Sheffield Hallam University.

The code for the sample size calculator is based on data from Raosoft Inc.