When doing quantitative market research, there are two ways
to select test respondents:
- probability samples (randomly selected samples)
- non-probability samples
With probability sampling, each test respondent sampled has
an equal chance of being selected for testing. This means
that test results have a better chance of being representative
of the entire target population.
For example, if we were to test how many America Online
members read this file, we could theoretically obtain a list
from AOL and randomly sample 400 members by mail, phone, or
e-mail to obtain a representative probability sample. (In
reality, AOL does not release this information.) If we were to
try to sample 400 AOL members outside a given computer store for
our survey, it would be a non-probability sample.
- in the probability sample, each respondent has an
equal chance of being tested and represents the total
demographic dispersion of AOL members
- in the non-probability sample outside the computer
store, it may be biased by including too many students,
businessmen, single vs. married people, etc., depending upon
the location of the store, day of the week, and time of day
Non-probability sampling can be biased!
Many small companies utilize only non-probability sampling
methods in their research. This may be due to budget constraints
or historical practice.
But the difference between probability and non-probability
methods can be significant. Only probability sampling provides a
true representation of the total target population, accurate
predictability, and distribution levels.
Non-probability sampling has built-in biases that cannot be
separated or measured. If a high degree of accuracy and
predictability is not required, as in early exploratory
stages of new product development, then "convenience"
non-probability sampling method might be acceptable.