Syntes: Steenbergen-Hu & Cooper (2013) — tutoring systems in K–12 maths classrooms: near-zero average gain over regular teaching

Källa: Steenbergen-Hu, S., & Cooper, H. (2013). A meta-analysis of the effectiveness of intelligent tutoring systems on K–12 students' mathematical learning. Journal of Educational Psychology, 105(4), 970–987. https://doi.org/10.1037/a0032447

Verified numbers (full text read 2026-10-07)

• Corpus: 26 reports (1997–2010), 34 independent samples, K–12 mathematics; most compared ITS with regular classroom instruction (20 of the 31 such studies were Cognitive Tutor, often used ~40 % of class time over a school year); a few compared with homework or human tutoring.
• Overall vs regular classroom instruction: unadjusted g = 0.05 (fixed, 95 % CI .02–.09, p = .005) and 0.09 (random, CI −.03 to .20, p = .136); pretest-adjusted g = 0.01 under both models (not significant). Authors: ITS "had no negative and perhaps a small positive effect".
• Outcome type (fixed effect): course-related measures g = 0.19 (CI .11–.27); standardised tests g = 0.02 (CI −.02 to .06); difference not significant under random effects.
• Timing (fixed effect): measured at end of school year g = .01 (CI −.03 to .04) vs earlier g = .25 (CI .14–.35); not significant under random effects.
• Robust moderators: larger effects for interventions shorter than a school year than for a year or longer; larger for general students than for low achievers.
• vs human tutoring: one small study (Beal et al. 2010, 25 sixth graders, small-group tutors): half-time ITS + tutor ≈ full-time tutor (g = .20, SE .39). Not a test of ITS vs one-to-one tutoring.
• Authors on VanLehn 2011: VanLehn's d ≈ 0.76 came from randomised experiments that held content constant (excluding Cognitive Tutor curriculum studies), across STEM and largely college students, using the largest-effect outcome per study; this meta-analysis included ecologically valid comparisons and averaged all outcomes.

Role for C5

Limits (weaken, 2026-10-07): the near-human-tutor result is from controlled same-content experiments; in routine K–12 maths classroom use over a term or a year, the average gain over regular teaching is near zero (and ≈ 0 on standardised tests and at end of year). Distinct from the Kulik & Fletcher 2016 standardised-test bound (that meta still found a median ~0.66 SD against conventional teaching). Not a kill: VanLehn 2011 and Ma et al. 2014 stand for controlled comparisons, and the meta includes curriculum comparisons in which the ITS was not the only difference.

Split (2026-10-08): the school-use result now belongs to C10 (everyday school use), where the 2026-10-07 ledger entry moved; C5 keeps only the point about VanLehn 2011's inclusion rules and the single small human-tutor comparison.