This post has been updated on June 11, 2020.
Whether it’s classroom or e-learning, training is costly in terms of money and time. So it is only natural for businesses to want to be able to quantify the value that training sessions bring to the organization. Of course, learning itself is never really over but a training cycle is deemed complete once its effectiveness has been evaluated and plans for the next steps on the learning path have been drawn.
It sounds fairly simple but in reality, it is not. The effectiveness of a training session is not only measured by the short quiz at the end of it or by what trainees say on the feedback form. These things matter but that is only the first step. Luckily, Donald Kirkpatrick created a training evaluation model that gives this process a clear structure.
According to the American scholar, evaluation of a training program should always start with level one, and then, as time and money resources allow, should move sequentially through levels two, three, and four. Information gathered from the prior level serves as a starting point for the next level’s assessment. In this way, as we get higher on the pyramid, we get a more precise measurement of the effectiveness of the training program.
Of course, as with most things, as we move up, more time and resources need to be consumed in order to get, in this case, the most accurate analysis.
The 1st level — Participant reactions
As the keyword implies, evaluation at this level takes a look at how participants in a particular training program feel and think about it. The purpose is to answer questions about the participants’ perceptions:
Did they enjoy it?
Was the material delivered relevant to their job?
This type of evaluation is often conducted at the end of the session by giving out short forms to fill in or, in the case of online training, it consists of an internet-based survey containing the same questions as the paper version. It is not rare though for classroom training to end with an online survey that participants can take on their tablets or smartphones — everybody’s got one these days, especially in the corporate world.
As Kirkpatrick explains, every program should at least have this degree of evaluation in order to gather some information that will lead to the improvement of the learning experience.
Furthermore, the participants’ immediate reactions (and the fact that they are asked to quantify them) have important consequences for learning — which is the second level of the evaluation. Even though a positive reaction is not a solid guarantee for learning, a negative one almost definitely annuls all possibility of it occurring.
The 2nd level — Learning
To get an idea about how much information has been acquired in a training program, level two evaluations frequently use tests that are conducted before the learning session takes place (pretests) and after it is completed (post-tests). It is kind of like those “before” and “after” pictures we get in advertising for miracle weight-loss products.
Assessing at this level takes the evaluation beyond mere learner satisfaction and tries to assess the extent to which employees have advanced in skills, knowledge, or competency. Measurement at this level is a lot more challenging and time-consuming than the one at level one.
The methods are also a lot more varied, ranging from individual testing, group assessments, and one to one discussions with experts on the subjects that were taught. Sometimes, learning and development specialists choose to test a control group of employees who did not participate in the training session to compare their performance against employees who did receive training.
The 3rd level — Transfer
This third level is designed to measure the transfer that has occurred in learners’ behavior due to the training program. Evaluating at this level tries to give a specific answer to the question
Are the newly acquired skills, knowledge, or competencies being used by the learner in the workplace?
For example, if we are talking about a complaint handling training program, the assessor would like to see if the methods that were taught in the session are being applied when it comes to real customer complaints. For most corporate trainers this level represents the truest assessment of a program’s effectiveness.
However, assessing change in behavior is hard to predict and to quantify. Some trainers choose to use action plans that participants fill in with what they would like to improve. Still, it takes continuous observation, even for three to four months after the training was completed.
It is the only way to get an accurate assessment of whether the learners have made permanent changes and improvements based on training or have fallen back into old habits shortly after.
The 4th level — Results
The last level takes a crack at assessing training in terms of business results. If level three is the true measurement of effectiveness for trainers, this last one is the measure of all measures for managers and executives. As Cuba Gooding Junior eloquently put it in Jerry McGuire: “Show me the money!”
If, as a result of the training there is an increase in production, better quality, growth in sales, a decrease in costs, less work-related accidents and, the sweetest one of all, a big ROI, the management is happy. From a business point of view, this is the main reason for a training program, yet everyone is fully aware that determining learning results in financial terms is very difficult to measure.
Basically, the bottom of the pyramid measures how content employees were with the learning experience, and the measurement at the top will determine how content management will be with the results of that experience. The measure of that will almost certainly be reflected in the learning and development budget.
Roxana is a learning and development professional with over 10 years experience in corporate training.