作为一名培训顾问，我最经常被问到的一个问题就是：“能够评估我的培训是否成功的最有效的途径是什么？”而针对这类问题，我总是回答：“你有多少时间呢？”
版权信息｜来源：ATD 原标题：Evaluating the Impact of Learning Programs
翻译：培训江湖 ID：ondemandchina2010
虽然我常常都笑着回答，但我的回答确实十分认真的。然后我会以更加严肃的语气询问对方一系列的问题来理解他们的真实需求：
问题清单不仅仅只有以上这些。但不幸的是，这些看似简单的问题并没有直截了当的答案。当涉及到学习项目评估的时候，并没有所谓的一刀切的通用有效评估方法。
在培训专家决定学习项目要采用何种评估方法的时候，他们要考虑诸多的变化因素与阻碍因素，这其中包括了组织目标及文化、利益攸关方及听众的期待、需求分析、效能分析、资源、时间、预算、数据搜集方法及工具，甚至还有我们在培训中传授的不同类型的知识与技能。
许多组织没有花费足够的时间来进行培训评估。在进行学习项目评估的时候，很多公司关注的是学员的反应与学习程度，并不是培训对业务成果造成的影响。根据《评估学习：评估有用的学习指标》一书所说，只有“在对199名人才发展专家的调查中发现，只有35%的组织会在学习项目在某种程度上造成的业务成果。”事实上，我碰到过很多培训专家对评估流程总是有所惧怕，原因不一而足。
我承认在我早期的职业生涯中，我也不怎么花时间去评估我的培训项目。
当我们培训部仅有5个人，而我们服务的对象却是10多个部门、1000多名员工，我们的 时间和资源都太有限了。我们也发现我们培训学员的许多技能，特别是软技能都很难找到量化的评估指标。然而，一旦我们切实地进行了培训评估之后，我们才开始看到这些项目对业务成果的影响以及对组织成长的贡献。
从中我们可以得知哪些培训有效哪些培训无效。这可以让我们有的放矢，集中资源，对没什么效果的项目进行必要的改进。而由于评估，领导层能够清晰地看到在培训项目导致在生产力、绩效、客户满意度、员工敬业度及人员流失率等量化指标上产生的变化，这会让他们变得更有兴趣投资其他的培训项目。
最后，我们在培训评估及评估方法上进行了更多的投资。评估有效地帮助我们证实了学习项目对整体组织及其员工产生的正面影响。我们的培训项目评估得越多，我们评估得就越来越早。我们越是提前制定好评估的计划，我们最后执行评估的时候花得时间也就越少。培训评估逐渐成为我们在培训流程中最喜欢的一部分，因为我们从中可以清楚地看到自己劳动的果实。
我们的评估流程非常全面、流畅，与许多学习项目因素与业务需求整合为一体。当我们发现这能够行得通的时候，我们就一直使用该模型，只根据我们想要培养的行为或想要改善的指标的不同，做出些微的调整。
下面列出了一些我们觉得很有帮助的几项要点，可供你们在评估自身学习项目时参考。
1、制定评估计划，进行需求分析
要做评估，我们不能把它当成马后炮。有效的培训评估必须要经过细致的计划与目标制定。这就意味着我们要认真思考以下几个问题：
该问题清单也不仅仅只有以上这些。此时，关键在于在开始设计及开发你的学习项目之前你就必须要先清楚地了解你要评估的内容、原因、方法、时间及人员，甚至是你的评估汇报对象。
需求分析的阶段则是后续学习项目的基础。特别是针对评估计划来说，需求分析与绩效分析都能够帮助我们有效地确定评估的诸多驱动因素数据，包括利益攸关方、目前绩效水平、期望绩效水平、差值、指标、基线等。
在你搜集数据的时候，你就可以开始规划评估的相关量化指标了。需求分析能够给你提供基线数据对照，以此来跟进学员的学习进展、行为改变、业务成果提升等，这些数据变化可以证实培训成功与否。
2、明确评估指标
在评估中你要明确的第一件事就是你要评估的指标是什么。
柯氏四级培训评估与菲利普斯ROI方法论能够帮助我们将学习项目分割为不同的层级，并确定相关的评估指标。而评估指标的层级越高，相关的信息也就越有价值，对组织整体来说也越重要。
通过预见及量化你对评估的期望值，你能够很清楚地看到自己的培训项目是否达到了目标。如果实际数据与你预期的数据不同，那么这就是你学习的机会，你可以去查证中间有什么出了差错，是什么原因导致的并找出下次能够改进的方法。
根据以上我们讨论的方法进行需求分析能够帮助你设计相关目标，而这些目标正好能够帮助你设计项目的学习目的，这也是评估学习项目是否有效的一个关键因素。
3、学习目的
有效学习项目的学习目的常常都与部门及整个组织的目标紧密相关。在问：“我们希望这个学习项目有什么成果？”的时候，你不仅仅单指了这个项目的学习目的，同时也确定了预期项目效果。
预期效果举例来说包括了客户满意度更高、销售额上涨、生产力提升、加班减少、瑕疵品减少、员工敬业度增高及人员流动降低。不管你的预期效果是什么，学习目的一定要具体、量化、可达成、有关联性、有时效性且与部门及组织目标紧密相关。
如果学习目的达成了，你就能够知道你的培训成功了。你也能看到你的培训为你的组织达成其业务目标做出了相关的贡献。
4、学习目的
一旦你弄清楚了具体的学习目的与项目预期成果，你的下一步就是设置具体的学习目标了。学习目标既要与学习目的一致，也要与组织目标紧密相关。
学习目标能够为你提供一种测量培训有效性的新方法。这种方法能够更细致地检测学员被要求掌握的知识、技能、行为及工作迁移应用等。
学习目标的ABCD模式能够帮助我们梳理编写有效的、可量化的学习目标。
如果能够准确地制定学习目标，不管是在学员在项目刚刚结束时，还是他们回到工作岗位几个月后，你都能够按照这些目标的达成程度来衡量你的培训是否成功。
可能有些人觉得奇怪，“我难道不应该在评估之前先确定学习目标与目的吗？”这其实要视情况来定。比起线性流程，评估学习项目的有效性更像是舞蹈前进，但都同样有着流程步骤。
正如《实用培训评估》一书中所说，“评估既是一门科学也是一门艺术。”尽管这看起来只是一种分析，但确实，评估所包含的内容非常广。要做到有效评估，我们必须先制定有效的评估计划、与利益攸关方进行良好合作、询问正确的问题、确定真实需求与绩效差距并就此与他人沟通、设计并开发相关的培训项目、基于评估制定后续的改善建议，这样才能全面无漏洞地检测学习项目的成功。
所以让我们回到最初的问题：“能够评估我的培训是否成功的最有效的途径是什么？”这个问题的答案就在上述内容中。如果你的时间或资源有限，只能去评估少数几个方面的话，那么评估学习项目对业务成果产生的影响将是最为重要的方面之一（主要取决于你的组织）。
不管你选择如何评估你的学习项目，有一件事是肯定的：随着人才培养领域不断地进化，评估将会成为所有学习及发展专家所面临的必须任务。有效评估将是保证未来学习项目需求的关键。
英文原文如下：
Evaluatingthe Impact of Learning Programs
Thursday, March 02, 2017 - by Emmy Monticelli
As a training consultant, one of the most frequentquestions I get asked is, “What is the most effective way to measure thesuccess of my training?” to which I reply, “How much time do you have?” with asmile and a laugh, but being completely serious. Then in a more serious tone,I’ll ask a series of questions to truly understand what they need:
What are youmeasuring?
What is thepurpose of your evaluation?
How do youdefine a successful learning program?
With whomare you sharing the results?
How you arecurrently evaluating the impact of your learning programs?
The list goes on. Unfortunately, there is nostraightforward answer to this seemingly simple question. When it comes toevaluating learning programs, there is no one-size-fits-allapproach that works best. There are many variables (and obstacles) training professionalsneed to consider when determining how to evaluate learning programs, includingthe organization’s goals and culture, stakeholders and audience’s needs, needsassessments, performance analysis, resources, time, budgets, data collectionmethods and tools, and even the different types of knowledge and skills we aretrying to impart.
Many organizations are not taking the time to effectivelyevaluate their training. When they do evaluate learning programs, mostcompanies focus on participants’ reactions and learning, not the impact it hason the business results. According to Evaluating Learning: Getting to Measurements That Matter, only “35 percent of 199 talentdevelopment professionals surveyed reported their organizations evaluated thebusiness results of learning programs to any extent.” In fact, many trainingprofessionals I speak to fear the evaluation process for one reason or another.
I will admit that early in my career, I didn’t take thetime to effectively evaluate my training programs either. With a total of fivepeople in the training department serving more than 10 departments and 1,000employees, we had limited time and resources. We also found it difficult toquantify many of the skills we were training, especially the soft skills.However, once we started evaluating our training, we began seeing how theyaffected business results and contributed to the organization’s growth.
We learned what worked and what didn’t. It allowed us torefocus our priorities and resources and make necessary changes to programsthat weren’t effective. We also began getting leadership’s buy-in for othertraining programs because they were seeing quantifiable measurements onproductivity, metrics, customer satisfaction, employee engagement, andturnover. Lastly, we were more invested in what we were doing and how we weredoing it. The evaluations helped validate that our learning programs weremaking a positive difference on the overall organization and its people. Themore we evaluated our training, the easier it was to measure. The more weplanned the evaluations ahead of time, the less time-consuming they were at theend. The evaluation ended up becoming one of our favorite parts of the processbecause we could see the fruits of our labor.
Our evaluation process was a very “holistic,” fluid onethat connected many of the learning program elements and business needs. Oncewe found what worked, we continued using the same model with slight variationsdepending on the behaviors we were trying to shape or metrics we wanted toimprove. Below are a few things to consider when evaluating your learning programsthat we found helpful.
1. Planning the Evaluation and Needs Assessment
When it comes to evaluations, we can’t treat them as anafterthought. Effective evaluations always begin with careful planning andpurpose. That means thinking about:
what youwant to learn and measure from the evaluationsthe datacollection methods and tools you will utilize
whom you aregetting the information from the timingof collecting and analyzing the data
whom you arereporting the information to and how. (Know your stakeholders! Meet with themprior to delivering the completed evaluation to ensure what and how you arepresenting is valuable to them.)
The list goes on. The point is to be purposeful in what,why, how, when, and whom you are evaluating, as well as to whom you arereporting before you even begin to design and develop your learning program.
The needs assessment phase serves as the basis for somany things in the learning program. Particularly for evaluation planning, aneeds assessment, along with performance analysis, helps us identifystakeholders, current performance, desired performance, gaps, metrics,baselines, and other important data that drives the evaluation. As you collectdata, you can even begin to formulate the quantifiable measurements of theevaluation. Needs assessments can provide baseline information to tracklearning, behavior changes, and increased business results that signify asuccessful training.
2. What Are You Measuring?
The first thing you need to be clear on is what you aremeasuring. For instance, do you want to know if someone liked your program ortransferred a new skill back on the job, if the training yielded increasedbusiness results, or the program’s return on investment (ROI)? These questionsshould be asked as part of your needs assessment, even before the design phase,because they help shape the overall learning program. The evaluation phase of alearning program doesn’t happen at the end of the program; it happensthroughout the entire process.
Kirkpatrick’s Four Levels of Evaluation and the Phillips ROI Methodology help identify specific measurements for learning programs broken down intodifferent levels. The higher the level, the more valuable the information andsignificance it has to the overall organization.
By anticipating and quantifying your expectations of theevaluations, you will be able to see if your learning programs are on target.If the actual numbers are different from your projected ones, that is anopportunity to learn what happened, why, and how to improve it for next time.Conducting the needs assessment as discussed above helps you project thesetargets. These targets also help to shape the learning goals of program,another critical element in effectively evaluating learning programs.
3. Learning Goals
Effective programs always start with learning goals thattie back to the department and overall organization’s goals. By asking, “Whatoutcome do we want as a result of this learning program?” you will not onlyisolate the program’s learning goals, but also identify the results.
Examples of desired results could include higher customersatisfaction ratings, increased sales, more productivity, less overtime, fewerdefects, more employee engagement, and less turnover. Whatever the desiredoutcome, learning goals need to be specific, measurable, achievable, relevant,time-based, and tie back to the department or organization’s goals.
Ultimately, once the learning goals are achieved, youwill know your training was successful. You will have also contributed inhelping your organization reach its business goals. One tip: Be sure to getleadership’s agreement on these goals at the very beginning before designingthe learning program. Getting everyone on the same page at the beginning savesyou a lot of time and energy in the end.
4. Learning Objectives
Once you know the specific learning goals or outcomes ofthe program, the next step is to create specific learning objectives, which tieback to the learning goals, which connect to organizational goals. Do you see apattern here?
Learning objectives provide another way to measure theeffectiveness of your training that is more detailed in the knowledge, skills,and behavior the participants are acquiring and applying on the job.
The ABCD format of Learning Objectives provides afour-component structure to writing effective, measurable objectives:
audience:who is being trained
behavior:what will the audience be doing or saying; this may include knowledge, skills,or attitudes
condition:resources, tools, or equipment used to complete the desired behavior
degree: howwell or how often the audience will be able to demonstrate the behavior.
If the learning objectives are donecorrectly, you could measure the success of your training once the audience hasmet these objectives either right after the program or a few months back on thejob.
Some of you may be wondering, “Shouldn’t I determine thelearning objectives and goals before deciding on what I am going to measure?”It depends. Evaluating the effectiveness of our learning programs is much moreof a dance than a linear, step-by-step process. As stated in Real World Training Evaluation, “Evaluation is both a science and an art.” Although itmay seem very analytical, there is so much more that goes into evaluations. Wehave to be able to effectively plan, work well with all stakeholders, ask theright questions, identify and communicate actual needs and performance gaps,design and develop relevant learning programs, and make future recommendationsbased on the evaluations to seamlessly measure the success of our learningprograms.
So back to the question, “What is the most effective wayto measure the success of my trainings?” The answer is “all of the above.” Ifyou only have time or resources to do a couple of things, evaluating how thelearning program contributed to the business results will probably be the mostsignificant (depending on your organization).
No matter how you choose to evaluate your learning programs, one thing isfor sure: As the landscape of talent development continues to evolve,evaluation is a necessary task for learning and development professionals.Effective evaluation is key to ensuring more learning programs in the future.
