People Analyst

Repeated Measures ANOVA

Repeated Measures ANOVA: What is it?

A repeated measures ANOVA (analysis of variance) is a statistical technique that is used to compare group means measured from the same group multiple times. It’s a great way to explore changes in a variable of interest over time. Talent management and Human resources departments may find this technique especially useful in evaluating the effectiveness of training programs or development initiatives. Here we will explore an example of an organization using a repeated measures ANOVA analysis.

Steps:

1. Define Your Variables Collect and Input Data:

The organization needs to find out how effective their three-week training program is at improving participants’ self-efficacy level. After training each week, scores on a self-efficacy measure are collected from the participants. This is what will be used to measure the effectiveness of the training sessions. The following table shows the data that will be used. repeatedmeasuresdata

2. Check Assumptions

Next, we make sure that the dataset does not violate any assumptions of normality, sphericity, and that there are no extreme outliers that could bias our results. The Shapiro-Wilk test of normality and Mauchly’s test for sphericity indicated that neither of these assumptions were violated. The QQ-plot below shows that the data is normally distributed.

repeatedmeasuresqqplot There was however one outlier in the second group, though it was not an extreme outlier so its effects on the final analysis will be negligible.

3. Run analysis

The results of the ANOVA show that the training sessions had a significant effect on the participants’ level of self-efficacy, F(2, 18) = 55.467, p < .001, 2ηG2 = 0.829. The rather large effect size indicates that approximately 82.9% of the variance in scores is accounted for by the training sessions. In post-hoc pairwise comparisons of scores across different points in time, significant increases in score were observed for each measurement, indicating a steady increase in level of self-efficacy after each training session. These training sessions have been shown to be highly effective in increasing levels of self-efficacy.

4. Conclusion

Repeated measures ANOVAs can be a great tool for evaluating training effectiveness. By looking at how interventions or other variables affect employees over time, employers can gain insight into the results of their initiatives.

This project was run in R using the following code:


dat1 <- A4_data1 %>%
  gather(key = "time", value = "score", t1, t2, t3) %>%
  convert_as_factor(id, time)
dat1
dat1 %>%
  group_by(time) %>%
  get_summary_stats(score, type = "mean_sd")
dat1 %>%
  group_by(time) %>%
  shapiro_test(score)
res.aov <- anova_test(data = dat1, dv = score, wid = id, within = time)
res.aov
bxp <- ggboxplot(dat1, x = "time", y = "score", add = "point")
bxp
dat1 %>%
  group_by(time) %>%
  identify_outliers(score)
ggqqplot(dat1, "score", facet.by = "time")
get_anova_table(res.aov)
pwc <- dat1 %>%
  pairwise_t_test(score ~ time, paired = TRUE,
    p.adjust.method = "holm")
pwc

Next Top