Do Intelligent Tutoring Systems Benefit K-12 Students? A Meta-Analysis and Evaluation of Heterogeneity of Treatment Effects in the U.S.
Primary school (grades 1–6) Lower secondary (grades 7–9) Upper secondary AI and learning Foundations of optimal learning
Links to claims
Supports (1)
| Claim | Basis |
|---|---|
| C10 | Meta-analysis of 18 US K-12 experiments meeting WWC standards: tutoring systems vs regular teaching g = 0.27 overall, 0.20 on standardised tests (0.166 to 0.234), 0.16 at the end of the school year, 0.11 (ns) for use longer than six months; half the effects from one reading tutor (ITSS). Kill-condition test, C10 survived 2026-11-15. Submitted version read. |
Review state: first-pass draft, pending human review.
Source note
Syntes: Leite et al. (2026) — intelligent tutoring systems in US K–12 experiments: g = 0.27 overall, 0.20 on standardised tests, 0.16 at the end of the school year
Källa: https://doi.org/10.1080/19345747.2026.2736856 (submitted version read: arXiv 2511.04997)
Verified content
- [full text] Search 2011–2021 (ERIC, PsycInfo, LearnTechLib and others), double-screened in Covidence; only experiments meeting WWC standards; 18 studies, 77 effects, 11 systems (pp. 13, 18–19).
- [full text] Systems: ITSS reading-structure tutor 39 effects (Wijekumar et al. 2012–2022), SWELL 10, MATHia 6, vocabulary ITS 6, ALEKS 6, Wayang Outpost 4, Cognitive Tutor Algebra I 2, and four with one effect each (Table 1).
- [full text] Overall g = 0.271 (SE 0.011, p = .001). Standardised tests 0.200 (0.166 to 0.234; 61 effects) vs researcher-made 0.483 (−0.154 to 1.120, ns); immediately after 0.460 (0.244 to 0.677) vs end of school year 0.155 (0.125 to 0.185; 47 effects); maths 0.253, reading and writing 0.323 (Tables 7, 9).
- [full text] Duration: up to two weeks 0.334 (ns), two weeks–three months 0.449 (ns), three–six months 0.225 (0.171 to 0.279), more than six months 0.110 (−0.077 to 0.297, ns). Main instruction 0.294, homework 0.317, separate activity 0.235 (ns), after-school 0.014 (ns) (Table 8).
- [full text] Elementary 0.266, middle 0.256; low achievers 0.270; rural 0.146 vs urban 0.298; journal articles 0.190 vs dissertations 0.625; fail-safe N (Orwin, target 0.10) 191.
Limits
- [full text] Not preregistered (OSF resources page only). Only 8 of 18 studies reported differential attrition.
- [assessment] Half the effects come from one reading tutor (ITSS) by one research group; not all systems are step-based maths tutors; US only; very narrow intervals from the robust-variance model.
Role for C10
- [assessment] Kill-condition test of C10 (survived 2026-11-15): on standardised tests and at the end of the year the gain over regular teaching is small but its interval lies above 0.1 SD; the longest-use subgroup is not significant.
Added 2026-11-15 by the project (nightly attack run). [full text] = read in the submitted version; [assessment] = the project's own assessment.
Metadata as recorded
- ID
- AI-050
- Tier
- meta
- Titel
- Do Intelligent Tutoring Systems Benefit K-12 Students? A Meta-Analysis and Evaluation of Heterogeneity of Treatment Effects in the U.S.
- Författare
- Leite, W. L.; Zhang, H.; Rana, S.; Hao, Y.; Busch, A. H.; Kong, L.; Kuang, H.
- År
- 2026
- Källa/DOI
- https://doi.org/10.1080/19345747.2026.2736856
- Tidskrift
- Journal of Research on Educational Effectiveness (online 2026-11-06)
- Status
- in the library since 2026-11-15 (AI-050; nightly attacks, C10 kill-condition test, survived)
- Åldersgrupp
- grundskola-lag;grundskola-hog;gymnasium
- Teman
- ai-och-larande;optimalt-larande-grunder
- Verifierad
- Crossref metadata (title, seven authors, JREE, published 2026-11-06) checked 2026-11-15
- Hämtad
- 2026-11-15
- Paywall
- published version closed from the box; submitted version open (arXiv)
The original files (PDF or abstract) are kept in the project archive and are not published here.