When doing quantitative market research, there are two ways to select test respondents:

• probability samples (randomly selected samples)
• non-probability samples

With probability sampling, each test respondent sampled has an equal chance of being selected for testing. This means that test results have a better chance of being representative of the entire target population.

For example, if we were to test how many America Online members read this file, we could theoretically obtain a list from AOL and randomly sample 400 members by mail, phone, or e-mail to obtain a representative probability sample. (In reality, AOL does not release this information.) If we were to try to sample 400 AOL members outside a given computer store for our survey, it would be a non-probability sample.

• in the probability sample, each respondent has an equal chance of being tested and represents the total demographic dispersion of AOL members
• in the non-probability sample outside the computer store, it may be biased by including too many students, businessmen, single vs. married people, etc., depending upon the location of the store, day of the week, and time of day

 

Non-probability sampling can be biased!

Many small companies utilize only non-probability sampling methods in their research. This may be due to budget constraints or historical practice.

But the difference between probability and non-probability methods can be significant. Only probability sampling provides a true representation of the total target population, accurate predictability, and distribution levels.

Non-probability sampling has built-in biases that cannot be separated or measured. If a high degree of accuracy and predictability is not required, as in early exploratory stages of new product development, then "convenience" non-probability sampling method might be acceptable.