Effect sizes in single-case aphasia studies: A comparative, autocorrelation-oriented analysis

Purpose: In single-case treatment studies, researchers may compare client performance during a baseline, nontreatment phase(s) to client performance during intervention phases. Autocorrelation in the data series gathered during such studies increases the likelihood that analysts will detect or fail to detect meaningful differences between baseline and treatment phase data. We examined the impact that autocorrelation has on 4 effect size calculation methods when these methods are applied to data generated by people with aphasia during anomia treatment studies. The effect sizes we selected were Busk and Serlin’s d, Young’s C, nonoverlap of all pairs, and Tau-U. We hypothesized that d and C would be influenced by autocorrelation, whereas nonoverlap of all pairs and Tau-U would not. Method: We extracted 173 highly autocorrelated data series from published investigations of treatments for anomia. These data series were then “cleansed” of autocorrelation through the use of an autoregressive integrated moving average (ARIMA) process. The 4 effect size calculation methods were used to derive an effect size for each published and each corresponding ARIMA-tized data series. The published and ARIMA-tized effect sizes associated with each calculation method were then compared. Results: For all of the 4 effect sizes, statistically significant differences existed between the published effect sizes and the ARIMA-tized effect sizes. Conclusions: All 4 of the methods were influenced by autocorrelation. Further research that develops effect size calculation methods that are not influenced by autocorrelation will help to improve the quality of single-case studies.