JUNE 2012 & 2013

JUN 2012 & JAN 2013

Question 4

C) Explain five (5) factors that influence the type of evaluation

    design

FACTOR	how the factor influences the type of evaluation design
change potential	can the program be modified?
importance	does ineffective training affect customer service, safety, product development or relationship among employees?
scale	how many trainees are involved?
purpose of training	is training conducted for learning, result, or both?
organization structure	is demonstrating results part of company norms and expectation?
expertise	can a complex study be analyzed
cost	is evaluation too expensive?
time frame	when is the information needed?

JUNE 2012 (1)

Jun 2012

Question 4

b)Discuss three reasons for evaluating training.

     i.        To identify the program’s strengths and weaknesses. This includes determining if the program is meeting learning objectives, if the quality of the learning environment is satisfactory, and if transfer of training to the job occurring.

    ii.        To assess whether the content, organization, and administration of the program. This includes the schedule, accommodations, trainers, and materials that contribute to learning and use of training content on the job. 

 iii.        To assist in marketing programs through the collection of information from participants about whether they would recommend the program to others, why they attend the program, and their level of satisfaction with the program.

a)    Describe two types of evaluation design.
 1.      Summative evaluation

•  Refers to an evaluation conducted to determine the extent to which trainees have changed as a result of participating in training program
• Involves collecting quantitative (numerical) data through tests, ratings of behavior, or objective measures of performance such as volume of sales, accidents, or patents.

  2.  Formative evaluation

• refer to an evaluation of training that take place during program design and development. 
• involves collecting qualitative data about the program that include opinions, belief and feelings of the customers, employees and managers on the description of the training content, program, design, and objectives about the program.

         

JUN 2012

4b) Describe two (2) types of evaluation design. (6 marks)

There are two types of evaluation design such as:

1) The posttest only

- The posttest-only design refers to an evaluation design in which only post-training outcomes are collected. The posttest-only design is appropriate when trainees (and the comparison group, if one is used) can be expected to have similar levels of knowledge, behavior, or results outcomes (eg: same number of sales, equal awareness of how to close a sale, prior to training).

2) Pretest/ Posttest

- The pretest/posttest refers to an evaluation design in which both pre-training and post-training outcome measure are collected. This design is often used by companies those wants to evaluate a training program but are uncomfortable with excluding certain employees or that only intend to train a small group of employees.

JAN 2012 & 2013

JANUARY 2012 & 2013

Question 5(b) & 4(b)

Discuss Kirkpatrick’s four-level framework for categorizing training outcomes (10 marks)

There are four level of Kirkpatrick’s framework for categorizing training outcomes. Level 1 is reaction. Reaction evaluation measures how the trainees felt about the training or learning experience. It also refers to trainees’ perceptions of the program, including the facilities, trainers and content. How to measure the reaction is by using feedback form or questionnaires.

Level 2 is learning or cognitive. Learning criteria is the measurement of the increase in knowledge before and after the training. The learning outcomes measures what trainees understand about the principles, facts and techniques. Company can measure the learning by using test before and after the training or make observations.

Level 3 is behavior. The behavior evaluation is how trainees applied learning back on the job implementation includes motor skills and behavior change. Observation and interview should be done continuously to measure the changes

Level 4 is the result. Result evaluation is the effect on the business and environment by the trainee. Company’s payoff can be measured by comparing business data on productions of employee before and after the training.

JAN 2012

Jan 2012

Question 5

a)With the aid of a diagram draw the evaluation process of training (5 marks)

APRIL 2011

APRIL 2011

Question 5

a) Give two (2) reasons why the Solomon Four-Group designs provide the most controls for internal and external validity? (10 marks)

The Solomon Four-Group designs provide the most control for internal and external validity because it combines the pretest/posttest comparison group and the posttest-only control group design. In the Solomon four-group design, a training group and a comparison group are measured on the outcomes both before and after training.

Pretest/posttest comparison group refers to an evaluation design that includes trainees and a corporation. If improvement is greater for the training group then the comparison group, this finding provides evidence that training is responsible for the change. This type of design controls is already known for most of the threats to validity as it provide a baseline or pretraining measure of the outcome. The comparison between posttraining and pretraining measures can indicate the degree to which trainees have changed a result of training.

The posttest-only design refers to an evaluation design in which only post-training outcomes are collected. The posttest-only design is appropriate when trainees (and the comparison group, if one is used) can be expected to have similar levels of knowledge, behavior, or results outcomes (eg: same number of sales, equal awareness of how to close a sale, prior to training)

OCT 2010 (1)

Oct 2010

Question 3

a)Discuss five reasons why organization should evaluate its training program.

 1.        To determine financial benefits and costs of the program

• To compare the costs and the benefits that the organization receive from the program
• The organization will gain benefits when the trainees can transfer the learning to the actual job.

 2.        To compare the costs and benefits of training versus nontraining investments

•   The organization must ensure that they receive more benefit once the trainees sent to the training program    compared to the non-training employees
• The trainees who involved in the training should contribute the learning to the organization so that training become effective and cost incur will be covered.

3. To compare the cost and benefits of different training programs to chose the best program.

• the trainer should evaluate which type of training program give more effectiveness to the trainees and the organization.

4. To identify which trainees benefits most or least from the program.

• the organization able to evaluate trainees that gain more benefits from the program
• organization can differentiate and rank employee according to their capability and level of understanding.

5. To assess whether the content, organization and administration of the program includes schedule, accommodation, trainers and materials contribute to learning and the use of training content on the job.

       

Oct 2010

Question 3

a)Discuss 5 reasons why organization should evaluate its training program. (15 marks)

1.    To identify the program’s strength and weaknesses.

-This includes determining if the program is meeting the learning objectives,if the quality learning environment is satisfactory and if transfer of training to the job is occurring.

2.    To assess whether the content, organization and administration of the program is contribute to learning and the use of training content on the job.

3.    To identify which trainees benefit most or least from the program.

-As different trainees have different knowledge,skills and capabilities,they may enhance each of their cognitive through the program.Therefore,as the superior in the organization,the manager may detect which is among his employee need more or least the training programs.

4.    To assist in marketing programs through the collection of information from participants about whether they would recommend the program to others,why they attended the program and their level of satisfaction with the program.

5.    To compare the costs and benefits of different training programs to choose the best program.

APRIL 2009

APRIL 2009

4a)  Explain the concept of threats to validity in the training evaluation. (8 marks)

Threats to validity refer to factors that will lead an evaluator to question either the believability of the study results or the extent to which the evaluation results are generalizable to other group of trainees and situations.

The believability of study refers to internal validity. The internal threats to validity relate to characteristics of the company (history), the outcome measures (instrumentation, testing), and the person in the evaluation study (maturation, regression toward the mean, mortality and initial group differences).

Threats to external validity relate to how study participants react to being included in the study and the effects of multiple types of training such as reaction to pretest, reaction to evaluation, interaction of selection and training and interaction of methods.