Investigation of Five Nonoverlap Methods to Calculate Effect Sizes in Single-Case Research
DOI:
https://doi.org/10.33308/26674874.2022362393Keywords:
Effect Size, Nonoverlap Methods, Single-Case Experimental Research, Tau-U, PND, Phi, IRDAbstract
Evidence-based practices (EBPs) include teaching methods and strategies aimed at improving learning outcomes of all children in schools. In recent years, movement to determine EBPs in education has expanded use of meta-analysis and led to development of effect size calculation methods in single-case experimental research (SCER). The purpose of this study was to examine five nonoverlap methods developed for effect size calculations in SCER. In this study, effect size calculations and visual analyzes were performed using five methods for each of 222 graphs obtained from SCER studies. Findings of the study revealed that four of the five methods (IRD, Tau-U, TauNovlap, and Phi) were highly correlated with each other and there were high levels of agreement between effect size estimates calculated using these methods and visual analysis. Considering their compatibility with visual analysis, it was determined that Tau-U and IRD performed slightly better than other methods.
Downloads
References
Allison, D. B., & Gorman, B. S. (1993). Calculating effect sizes for meta-analysis: The case of the single case. Behavior Research and Therapy, 31(6), 621–631. https://doi.org/10.1016/0005-7967(93)90115-B
Baer, D. (1977). Perhaps it would be better not to know everything. Journal of Applied Behavior Analysis, 10(1), 167-172. https://doi.org/10.1901/jaba.1977.10-167
Beretvas, S. N., & Chung, H. (2008). A review of single-subject experimental design meta-analyses: Methodological issues and practice. Evidence-Based Communication Assessment and Intervention, 2(3), 129-141. https://doi.org/10.1080/17489530802446302
Biosoft. (2004). UnGraph for Windows (Version 5.0). Author.
Bouck, E. C., Long, H., & Park, J. (2021). Using a virtual number line and corrective feedback to teach addition of integers to middle school students with developmental disabilities. Journal of Developmental & Physical Disabilities, 33(1), 99-116. https://doi.org/10.1007/s10882-020-09735-z
Büyüköztürk, Ş., Kılıç Çakmak, E., Akgün, Ö. E., Karadeniz, Ş., & Demirel, F. (2020). Eğitimde bı̇lı̇msel araştırma yöntemleri. Pegem Akademi. https://doi.org/10.14527/9789944919289
Campbell, J. (2004). Statistical comparison of four effect sizes for single-subject designs. Behavior Modification, 28(2), 234-246. https://doi.org/10.1177/0145445503259264
Campbell, J. (2013). Commentary on PND at 25. Remedial and Special Education, 34(1), 20-25. https://doi.org/10.1177/0741932512454725
Center, B. J., Skiba, R. J., & Casey, A. (1985-86). A methodology for the quantitative synthesis of intra-subject design research. Journal of Special Education, 19(4), 387-400. https://doi.org/10.1177/002246698501900404
Cook, B. G., Tankersley, M., & Landrum, T. J. (2009). Determining evidence-based practices in special education. Exceptional Children, 75(3), 365-383. https://doi.org/10.1177/001440290907500306
DeProspero, A., & Cohen, S. (1979). Inconsistent visual analysis of intrasubject data. Journal of Applied Behavior Analysis, 12(4), 573-579. https://doi.org/10.1901/jaba.1979.12-573
Gage, N. A. & Lewis, T. J. (2012). HLM-meta-analysis of single-subject design research. Journal of Special Education. https://doi.org/.1177/0022466912443894
Gulboy, E., Yucesoy-Ozkan, S., & Rakap, S. (2021). Embedded instruction in early childhood: A systematic review and meta-analysis of single-case experimental research studies. Under Review.
Hedges, L. G., Pustejovsky, J., & Shadish, W. R. (2012). A standardized mean difference effect size for single-case designs. Research Synthesis Methods, 3(3), 224-239. https://doi.org/10.1002/jrsm.1052
Huitema, B. E., & McKean, J. W. (2000). A simple and powerful test for autocorrelated errors in OLS intervention models. Psychological Reports, 87(1), 3-20. https://doi.org/10.2466/pr0.2000.87.1.3
Kazdin, A. E. (1982). Single-case research designs: Methods for clinical and applied settings. Oxford University Press.
Klingbeil, D. A., Van Norman, E. R., McLendon, K. E., Ross, S. G., & Begeny, J. C. (2019). Evaluating Tau-U with oral reading fluency data and the impact of measurement error. Behavior Modification, 43(3), 413-438. https://doi.org/10.1177/0145445518760174
Kratochwill, T. R., Hitchcock, J., Horner, R. H., Levin, J. R., Odom, S. L., Rindskopf, D. M., & Shadish, W. R. (2010). Single-case designs technical documentation. http://ies.ed.gov/ncee/wwc/pdf/wwc_scd.pdf.
Kratochwill, T. R., & Levin, J. R. (Eds.). (1992). Single-case research design and analysis: New directions for psychology and education. Erlbaum.
Kratochwill, T., & Levin, J. (2006). Single-case research design and analysis: Applications in educational intervention research. http://ies.ed.gov/funding/grantsearch/details.asp?ID=554
Ma, H. H. (2006). An alternative method for quantitative synthesis of single-subject researches: Percentage of data points exceeding the median. Behavior Modification, 30(5), 598-617. https://doi.org/10.1177/0145445504272974
Maggin, D. M., Briesch, A. M., & Chafouleas, S. M. (2013). An application of the What Works Clearinghouse standards for evaluating single-subject research: Synthesis of the self-management literature-base. Remedial and Special Education, 34(1), 44-58. https://doi.org/10.1177/0741932511435176
Maggin, D. M., Swaminathan, H., Rogers, H. J., O’Keeffe, B. V., Sugai, G., & Horner, R. (2011). A generalized least squares regression approach for computing effect sizes in single-case research: Application examples. Journal of School Psychology, 49(3), 301-321. https://doi.org/10.1016/j.jsp.2011.03.004
Manolov, R., & Solanas, A. (2013). A comparison of mean phase difference and generalized least squares for analyzing single-case data. Journal of School Psychology, 51(2), 201-215. https://doi.org/10.1016/j.jsp.2012.12.005
Odom, S., Brantlinger, E., Gersten, R., Horner, R. H., Thompson, B., & Harris, K. R. (2005). Research in special education: Scientific methods and evidence-based practice. Exceptional Children, 71(2), 137-148. https://doi.org/10.1177/001440290507100201
Park, H., Marascuilo, L., & Gaylord-Ross, R. (1990). Visual inspection and statistical analysis in single-case designs. Journal of Experimental Education, 58(4), 311–320. https://doi.org/10.1080/00220973.1990.10806545
Parker, R. I., & Brossart, D. F. (2003). Evaluating single-case data: A comparison of seven statistical methods. Behavior Therapy, 34(2), 189-211. https://doi.org/10.1016/S0005-7894(03)80013-8
Parker, R. I., & Brossart, D. F. (2006). Phase contrasts for multi-phase single case intervention designs. School Psychology Quarterly, 21(1), 124–136. https://doi.org/10.1521/scpq.2006.21.1.46
Parker, R. I., & Hagan-Burke, S. (2007). Median-based overlap analysis for single case data: A second study. Behavior Modification, 31(6), 919-936. https://doi.org/10.1177/0145445507303452
Parker, R. I., Hagan-Burke, S., & Vannest, S. (2007). Percentage of all nonoverlapping data (PAND): An alternative to PND. Journal of Special Education, 40(4), 194-204. https://doi.org/10.1177/00224669070400040101
Parker, R. I., & Vannest, K. J. (2009). An improved effect size for single case research: Nonoverlap of all pairs (NAP). Behavior Therapy, 40(4), 357-367. https://doi.org/10.1016/j.beth.2008.10.006
Parker, R. I., Vannest, K. J., & Brown, L. (2009). The improvement rate difference for single-case research. Exceptional Children, 75(2), 135–150. https://doi.org/10.1177/001440290907500201
Parker, R. I., Vannest, K. J., & Davis, J. L. (2011). Effect size in single-case research: A review of nine nonoverlap techniques. Behavior Modification, 35(4), 303-322. https://doi.org/10.1177/0145445511399147
Parker, R. I., Vannest, K. J., Davis, J. L., & Sauber, S. B. (2011). Combining non-overlap and trend for single case research: Tau-U. Behavior Therapy, 42(2), 284-299. https://doi.org/10.1016/j.beth.2010.08.006
Parsonson, B. S, & Baer, D. M. (1978). The analysis and presentation of graphic data. T. Kratchowill (Ed.), Single subject research içinde. Academic Press.
Pyle, K., & Fabiano, G. A. (2017). Daily report card intervention and attention deficit hyperactivity disorder: A meta-analysis of single-case studies. Exceptional Children, 83(4), 378-395. https://doi.org/10.1177/0014402917706370
Rakap, S. (2016). Özel eğitimde bilimsel dayanaklı uygulamalar. V. Aksoy (Ed.), Özel eğitim (ss. 181-211) içinde. Pegem Akademi.
Rakap, S., & Parlak-Rakap, A. (2011). Effectiveness of embedded instruction in early childhood special education: A literature review. European Early Childhood Education Research Journal, 19, 79-96. https://doi.org/10.1080/1350293X.2011.548946
Rakap, S., Snyder, P., & Pasia, C. (2014). Comparison of nonoverlap methods for identifying treatment effect in single-subject experimental research. Behavioral Disorders, 39(3), 128-145. https://doi.org/10.1177/019874291303900303
Rakap, S., Yücesoy-Özkan, Ş., & Kalkan, S. (2020). Tek-denekli Deneysel Araştırmalarda Etki Büyüklüğü Hesaplama: Örtüşmeyen Veriye Dayalı Yöntemlerin İncelenmesi. Türk Psikoloji Dergisi, 35(85), 40-60. https://doi.org/10.31828/tpd1300443320181023m000015
Salzberg, C. L., Strain, P. S., & Baer, D. M. (1987). Meta-analysis for single-subject research: When does it clarify, when does it obscure? Remedial and Special Education, 8(2), 43-48. https://doi.org/10.1177/074193258700800209
Scruggs, T. E., & Mastropieri, M. A. (2013). PND at 25: Past, present, and future trends in summarizing single-subject research. Remedial and Special Education, 34(1), 9-19. https://doi.org/10.1177/0741932512440730
Scruggs, T. E., Mastropieri, M. A., & Casto, G. (1987). The quantitative synthesis of single-subject research: Methodology and validation. Remedial and Special Education, 8(2), 24-33. https://doi.org/10.1177/074193258700800206
Shadish, W., & Rindskopf, D. (2010). A d-estimator for single case designs. http://ies.ed.gov/funding/grantsearch/details.asp?ID=953
Snyder, P., Rakap, S., Hemmeter, M. L., McLaughlin, T., Sandall, S., & McLean, M. (2015). Naturalistic instructional approaches in early learning: A systematic review of the empirical literature. Journal of Early Intervention, 37, 69-97. https://doi.org/10.1177/1053815115595461
Ugille, M., Moeyaert, M., Beretvas, S., Ferron, J., & Van Den Noortgate, W. (2012). Multilevel meta-analysis of single-subject experimental designs: A simulation study. Behavior Research Methods, 44(4), 1244-1254. https://doi.org/10.3758/s13428-012-0213-1
Van Den Noortgate, W., Beretvas, N., & Ferron, J. (2011). Multilevel synthesis of single-case experimental data: Further developments and empirical validation. http://ies.ed.gov/funding/grantsearch/details.asp?ID=1121
Van den Noortgate, W., & Onghena, P. (2003). Hierarchical linear models for the quantitative integration of effect sizes in single case research. Behavior Research Methods, Instruments & Computers, 35(1), 1-10. https://doi.org/10.3758/BF03195492
Vannest, K. J., Parker, R. I., & Gonen, O. (2011). Single case research: Web based calculators for SCR analysis (Version 1.0) [Web-based application]. College Station: Texas A&M University. www.singlecaseresearch.org.
Wolery, M., Busick, M., Reichow, B., & Barton, E. (2010). Comparison of overlap methods for quantitatively synthesizing single-subject data. Journal of Special Education, 44(1), 18-28. https://doi.org/10.1177/0022466908328009
Yucesoy‐Ozkan, S., Rakap, S., & Gulboy, E. (2020). Evaluation of treatment effect estimates in single‐case experimental research: comparison of twelve overlap methods and visual analysis. British Journal of Special Education, 47(1), 67-87. https://doi.org/10.1111/1467-8578.12294
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2022 Journal of Education for Life
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
I accept that the Owner of Journal of Education for Life, the Editor, Associate Editors, Reviewers and the Editorial Board cannot be hold responsible regarding the scope, the findings, the discussion and conclusion of the manuscript submitted.
I declare to the editorship of Journal of Education for Life that the manuscript is original and has not been published anywhere else or is not under evaluation process for any other journal.
I approve that I grant Journal of Education for Life as the sole and exclusive right and license to publish for the full legal term of copyright of my manuscript concurring with article 5846 / 22-23-25 while I retain copyright in the work.
