![Results
In order to test the hypotheses, three separate analyses of covariance (ANCOVAs) were
conducted; one for Weeks 4, 8, and 12. The first ANCOVA [between-subjects factor: Week4;
covariate: Week0] revealed no main effects of training, F (1, 144) = .43, p = .52, ηp
2 = .003.
However, there was a main effect for the training group variable, F (1, 144) = 92.06, p = .072,
ηp
2 = .57. After four weeks, the scores had an unexpected trend. The score for the employees in
group 1 (the only group to receive training) increased (M = 39.57, SE = 1.17), though not at a
significant level from the original mean (M = 12.47). However, the scores for employees in
group 3 (M = 45.21, SE = 1.17) also increased at a similar rate despite the fact that they had not
been trained during the initial four week period. Both groups score were significantly higher than
the scores of employees in the second group (M = 23.67, SE = 1.16).
The second ANCOVA [between-subjects factor: Week8; covariate: Week0] revealed no
main effects of training, F (1, 144) = .54, p = .46, ηp
2 = .004. There was a main effect for the
training group variable, F (1, 144) = 497.56, p = .000, ηp
2 = .88. The trend after eight weeks was
what I expected initially. The first group (M = 45.06, SE = .70) and the second group (M = 45.48,
SE = .69) had increases in their knowledge scores from the scores obtained before training,
which was expected because at the eight week mark both groups had received training. The third
group (M = 18.37, SE = .70) had scores that I anticipated before the analyses, but is problematic
because of their scores they had after four weeks (M = 45.21, SE = 1.17). There was an unusual
effect occurring in the third group that makes the results from the training a bit concerning.
For the third ANCOVA, [between-subjects factor: Week12; covariate: Week0] revealed
no main effects of training, F (1, 144) = 1.10, p = .30, ηp
2 = .008, or for the training group
variable, F (1, 148) = .03, p = .97, ηp
2 = .000. The trends were close to what I expected after](https://image.slidesharecdn.com/evaluationassignment5paper-150504115331-conversion-gate02/85/Evaluation-assignment-5-paper-2-320.jpg)
Evaluation assignment 5 paper
- 1. TECO has conducted training and evaluated the training to see if it is something that the organization should invest future resources into. With any training, there are costs associated with the implementation of training and as a result, it is imperative for TECO to wisely spend their resources on training that will be effective. Money is a scarce resource for organizations and because of that; if the training is determined to be ineffective (or even if the results are inconclusive) then it would probably be wise for TECO to invest their resources in more fruitful areas. I hypothesized that the self-report knowledge scores for employees would increase in the assessment after their group was trained. For example, the increase in scores for participants in group 1 should come after Week 4, but that there should not be a major score increase for those in groups 2 and 3. I also hypothesized that the supervisor performance ratings would be higher after year two than they were for year one. I also expected the year one performance ratings to be higher than the ratings conducted prior to training. The first step in determining the effectiveness of the training was to clean the data. After going through the 149 cases, decided to remove four particular cases from the data. I removed cases that had a substantial amount of missing information from either the sales report knowledge scale scores or the supervisor performance ratings. I considered a score of zero for any of those variables to be indicative of a missing score. Though there is a possibility that a score of zero could be a true score of zero, my thoughts were that a person who scores a zero would likely not be retained anyways, which would lead to missing data in their scores after the initial zero. I thought about removing cases where scores dramatically increased from one time point to the next without being previously trained, but most of the participants from groups 2 and 3 would have been excluded from the analyses.
- 2. Results In order to test the hypotheses, three separate analyses of covariance (ANCOVAs) were conducted; one for Weeks 4, 8, and 12. The first ANCOVA [between-subjects factor: Week4; covariate: Week0] revealed no main effects of training, F (1, 144) = .43, p = .52, ηp 2 = .003. However, there was a main effect for the training group variable, F (1, 144) = 92.06, p = .072, ηp 2 = .57. After four weeks, the scores had an unexpected trend. The score for the employees in group 1 (the only group to receive training) increased (M = 39.57, SE = 1.17), though not at a significant level from the original mean (M = 12.47). However, the scores for employees in group 3 (M = 45.21, SE = 1.17) also increased at a similar rate despite the fact that they had not been trained during the initial four week period. Both groups score were significantly higher than the scores of employees in the second group (M = 23.67, SE = 1.16). The second ANCOVA [between-subjects factor: Week8; covariate: Week0] revealed no main effects of training, F (1, 144) = .54, p = .46, ηp 2 = .004. There was a main effect for the training group variable, F (1, 144) = 497.56, p = .000, ηp 2 = .88. The trend after eight weeks was what I expected initially. The first group (M = 45.06, SE = .70) and the second group (M = 45.48, SE = .69) had increases in their knowledge scores from the scores obtained before training, which was expected because at the eight week mark both groups had received training. The third group (M = 18.37, SE = .70) had scores that I anticipated before the analyses, but is problematic because of their scores they had after four weeks (M = 45.21, SE = 1.17). There was an unusual effect occurring in the third group that makes the results from the training a bit concerning. For the third ANCOVA, [between-subjects factor: Week12; covariate: Week0] revealed no main effects of training, F (1, 144) = 1.10, p = .30, ηp 2 = .008, or for the training group variable, F (1, 148) = .03, p = .97, ηp 2 = .000. The trends were close to what I expected after
- 3. twelve weeks. All three groups produced scores that were higher than the initial pre-training knowledge scores. Though the results were not statistically significant, I believe that they are practically significant because of the knowledge increases that were found in the data. For the figure, please see Appendix. To summarize, some of the results were unexpected which is problematic because group knowledge gains may not be associated with training. This would suggest that other confounding factors may have affected the knowledge scores of the employees through the observations. This would further suggest that the training is not as effective as it may appear to be. Supervisor ratings began around the median (M = 54.68, SD = 15.01), significantly increased after one year (M = 85.63, SD = 8.40), but then regressed back toward the initial mean after two years (M = 56.17, SD = 14.70). This suggests that the training did not produce long term changes in the employee performance ratings after two years. TECO will have to determine whether the one year effects are worth the long-term investment. It is my belief that there were some other explanations for the scores seen in the performance ratings. Based on the scores, I believe that the training increased performance due to an increased awareness on the part of the employees. The training may have been seen by the employees as a final step by management before demotions and terminations begin. Regardless of the reason, whatever factor was causing the score jumps were not producing long term changes in employee performance. Annual profits related to the sales associates have increased each year after the training. Before the training, the profit related to sales associates was $1.5 million. One year after training, the profits increased to $1.75 million. After two years after training, the profits rose to 2 million. There is a steady increase in profits over the two year period, but I believe that could be the result of other factors. For instance, it could be the result of more employees being hired in sales
- 4. associate positions. Overall, employee reactions of the training were fairly positive (M = 7.12, SD = 1.90). Most of the results were consistent with what I anticipated before running the analyses, but the random increase in performance by group three after the first four weeks was shocking. It is my belief that either the members in group one decided to share what they learned in training to members of group three, or there may be reliability concerns in the self-report knowledge scale. I would not recommend using the training in the future because of the possible measurement reliability issues. I also would not suggest using this training because the performance results also only lasted for one year before diminishing back to their pre-training levels of performance. In considering the cost of administration, I am not convinced that it is worth the cost for a training that likely will not produce long term results. These are resources that I believe could either be spent in a more effective training or in targeting more knowledgeable prospects for the selection process. It is my belief that selecting more qualified applicants would solve many of the performance issues faced by TECO sales associates. However, if TECO decides to continue the usage of this training, I have a few suggestions for the organization. The first suggestion would be for TECO to create more than one training session that covers different levels of sales associate knowledge. For instance, the first training session could be an introductory class about the job and skills that can be used. The second session could cover intermediate concepts and be done a month after the first training session. Finally, a third session could be done two months after the second session and cover advance topics and job knowledge. This would increase the cost of training per employee, but I believe that it would lead to better performance over a longer period. Along those lines, I would
- 5. also suggest that TECO invests money into hiring a sales expert would could train the employees in the three different sessions over the two month period.
- 6. Appendix Week 12 Results Source Type III Sum of Squares df Mean Square F Sig. Partial Eta Squared Corrected Model 12.705 3 4.235 .408 .747 .009 Intercept 38414.172 1 38414.579 3701.579 .000 .963 Week 0 11.462 1 11.462 1.104 .295 .008 TrainingGroup .546 2 .273 .026 .974 .000 Error 1463.267 141 10.378 Total 295281.000 145 Corrected Total 1475.972 144
- 7. SPSS Tables