Survey Methodology

Facts, Myths, and Misunderstandings

1. "Random" means each individual element of the population being surveyed has an equal chance of being chosen. It does not have the same meaning as "arbitrary", in fact, just the opposite. A random sample must be drawn sytematically, rather than by convenience.

2. The needed sample size is not related to the size of the population. This is counterintuitive, but easier to accept if you consider that national polls Gallop poles, and exit poles, for example, a very small portion of the population. Sample size should be calculated using a formula which takes into account the desired precision ("confidence interval") and confidence "p value"), as well as the variability within the population.

3. A cluster survey is less accurate than a systematic random sample. It's for that reason that the formula for cluster sample size often doubles the sample size (a 2 for "design effect") to compensate for the shortcoming. A cluster sample, however, is often the only feasible sample design, when the population is quite large, or when there is no complete list ("sampling frame")of the population to be surveyed.

4. Survey results must report both confidence, and precision. Without that information, it is impossible to say how close the estimates may be. When estimating proportions, the confidence interval (precision)is narrower when a proportion is vary high, or very low; and wider as proportions approach 50%.