MAKING IT USEABLE    Interpreting Estimates       Sampling Bias

Sample bias has the following attributes:

• Sample bias does not decrease with sample size and may even increase, depending on the source of the bias.
• Sample bias can even be present in a census (a 100 % survey), if it arises from measurement problems and instrument problems.
• Sample bias cannot be calculated in most cases and bears no relation to sample size, population size, or variability of the measures being collected.

Sample bias may arise from a large variety of sources, including, but not limited to:

• Failure to adhere to the random sampling procedures.
• Omission of specific subgroups of the population from the sampling frame and therefore from the sample.
• Faulty measuring devices (this may be in terms of the specific questions used in a questionnaire, and may also arise in a survey that involves taking physical measurements, when the measuring device is incorrect, e.g., using a tape measure that has been stretched, so that all measurements are too small).
• Violations of equal probability of selection principles because of duplicate listings in the sampling frame, or other causes.
• Non-response to a survey by specific subgroups of the population that are relevant to the measures of concern in the survey.

In surveys of human populations, some bias is inevitably present, because of such issues as nonresponse and because it is impossible to design a survey questionnaire or interview that is completely free of bias. Even though measures were taken to minimize bias in surveys, bias is present in the Decennial Census conducted in the U.S. and is present in every other survey conducted on any segment of the population.

In surveys such as the NPTS, the biases that exist are carefully minimized and the sources are clear and understood, even though the magnitude of the bias cannot be estimated. The principal source of bias in the NPTS arise from:

• Inability to reach some households that are selected for the sample.
• Refusals by persons to participate in a household where most members do participate.
• Refusals to answer certain questions.
• Exclusion of non-telephone households.
• Effects of multiple telephone line households, and households sharing one telephone line.
• Refusals by households to participate.
• Use of questions that may be misunderstood by some minority of the sample and are answered incorrectly.

These sources of bias are either accepted or minimized as described below:

• Exclusion of non-telephone households is accepted as a bias in the NPTS, because methods to include such households would be very expensive in comparison to what is believed to be modest bias.
• Multiple telephone lines are corrected for by determining incidence of these in the sample and adjusting the weights on such households to account for the changed probability of sampling them.
• Refusals to participate or to answer specific questions are reduced through various efforts of persistence, use of incentives, etc., and inability to reach households is reduced by making a large number of attempts at different times on different days of the week. Nevertheless, the NPTS data user should be aware that low response rates are a concern and are comparable to other contemporary household surveys. A discussion of the specific response rates can be found in Section 3-E of the Users Guide.
• Bias from question wording is minimized through careful design of survey questions, pilot testing, and analysis of pilot-test results and interviewer feedback that can reveal problems with understanding of questions.

---