The Holy Grail: Capturing Elusive Level 3 Evaluation Data

Ken Phillips

Phillips Associates, USA

Are you one of the many L&D professionals who would like to conduct Level 3 evaluations but don’t? You’re not alone. According to a recent ATD research study, only 54 percent of organizations evaluate some learning programs at Level 3. To establish real credibility and prove value, you need to measure whether or not participants applied what they learned in a training program back on the job (Level 3 evaluation). Unfortunately, measuring on-the-job behavior change is not something many L&D professionals know much about nor often do. Also, some L&D professionals who are conducting Level 3 evaluations have fallen behind recent advances in evaluation and survey design methodology. Consequently, they likely are collecting data that is either misleading or invalid.

In this session, you will learn practical tips and guidelines on how to create Level 3 participant surveys that measure job behavior change. Particular emphasis will focus on three crucial areas of survey design: the content of the survey items, formatting of the survey, and the measurement scale used.

After attending this session, you will be able to:
• Use facts from a recent ATD research report to benchmark Level 3 evaluation use in your organization or a client organization.
• Avoid common mistakes that result in the collection of misleading or invalid data when creating Level 3 evaluation surveys.
• Use 12 tips and best practices for creating valid, scientifically sound Level 3 evaluations.