Sunday, July 25, 2021

Yes! All of our programs are available live-online. Contact us for details.

Written by Dr. Ken Reynolds of SRA International, Inc.

Random sampling is the basis for most of the procedures of statistical inference. An appropriate sampling methodology ensures the equal likelihood of drawing another sample with the same characteristics as the previous sample, even though both samples may not be exact copies of the population. Thus, what becomes more important in the applied world is that you approach sample selection in the same manner every time in order to minimize both statistical bias and the cost of sample selection as much as possible. 

For example, assume that you want to collect information through focus groups. This begs the questions of how many focus groups, the number of attendees, and so on.  In most cases, the best one can do regarding random sample selection is to ensure that the approach used to determine the number of focus groups and attendees is the same across the board. This is the best way to ensure that all focus group attendees have an equal likelihood of attending regardless of their location, and that the likelihood of maximizing participation across the focus group meetings is the same.

Concerning sample size, there is no standard sample size. When considering sample size and the power of statistical inference, determining the sample size should depend upon the consequences of coming to a wrong conclusion and the cost of adding more samples. In today’s resource-constrained market, the cost of increasing the sample size usually dictates (or at least heavily influences) the sample size relative to all other considerations.

Thus, a common solution evolving in today's business environment is to develop a random sample approach that will ensure the equal likelihood of drawing another sample with the same characteristics as the previous sample, and then to attempt to maximize the number of participants for each sample given cost and other constraints. Monte Carlo simulations have shown that sample sizes of 30 or more approach a normal (Gaussian) distribution, which is required to conduct statistical inference.

Please log in to ask a question below, or to leave a comment about sampling methods that have worked for you.

Additional Resources
Kirkpatrick Four Levels® Evaluation Certification Program
Kirkpatrick Four Levels® Evaluation Certificate Program
Posted in: Quick Tips
Actions: Permalink | Comments (2) |


Wednesday, April 3, 2013 12:29 PM
I think sampling is an important consideration for training evaluation. In addition to the thinking behind sample size, the very concept of sampling can help training evaluators consider ways to bring a daunting challenge down to reasonable, manageable, size.

You can find articles for guidance on selecting statistically valid sampling plans on the Internet. If you are interested in sampling, learning about how "Sampling Plans" work, and related producer risk and consumer risk, is eye-opening.

Still note that industrial sampling practices are influenced by feasibility. For example, a recommended sampling plan might entail checking 17 items for every lot of 125. There may only be time, or physical capacity, to check 12. Sometimes, 12 will become the sample size as agreed upon by the producer (the training coordinator) and the customer (training impact stakeholders and perhaps regulatory bodies).
# Wendy Partners
Wednesday, April 3, 2013 5:46 PM
Hi Doug,

Thank you for sharing your expertise with the community, and for providing ideas for how people can learn more on this topic!

Wendy Kirkpatrick
eLearning Learning

Copyright 2009 - 2021 Kirkpatrick Partners, LLC. All rights reserved.