Effectiveness of Cognitive Tutor Algebra I at Scale
Lower secondary (grades 7–9) Upper secondary AI and learning Foundations of optimal learning
Links to claims
Limits (1)
| Claim | Basis |
|---|---|
| C10 | Cluster-randomised at-scale trial of Cognitive Tutor Algebra I: no first-year gain; second-year high-school gain about +0.2 SD (p = .03; +0.21 in the authors' stricter re-analysis), middle schools ns. C5 weakened 2026-10-08 (school-use bound qualified). Moved from C5 to C10 at the split on 2026-10-08. |
Review state: first-pass draft, pending human review.
Source note
Syntes: Pane et al. (2014) — Cognitive Tutor Algebra I at scale: no gain in year 1, about +0.2 SD for high schools in year 2
Källa: Pane, J. F., Griffin, B. A., McCaffrey, D. F., & Karam, R. (2014). Effectiveness of Cognitive Tutor Algebra I at scale. Educational Evaluation and Policy Analysis, 36(2), 127–144. https://doi.org/10.3102/0162373713507480
Verified numbers (RAND addendum WR-1050 read in full 2026-10-08; published article closed, abstract read)
- Design (abstract): schools in seven states matched into similar pairs and randomly assigned to continue their algebra curriculum for two years or adopt Cognitive Tutor Algebra I (personalised, mastery-learning, blended); implemented under conditions similar to independent adopters; outcome an algebra proficiency exam.
- Published estimates (reproduced in the addendum, Tables A.1–A.2): high schools year 1: −0.19 (p = .10), −0.12 (p = .24), −0.10 (p = .34); year 2: +0.14 (p = .24), +0.19 (p = .05), +0.22 (p = .03) for Models 1–3 (Model 3 preferred). Middle schools year 1: −0.20, 0.00, −0.03; year 2: −0.07, +0.17, +0.19 (all ns).
- Addendum re-analysis (actual pretests, late joiners excluded): high schools 11,066 pupils (attrition 36.4 % / 27.7 %, differential 5.8 % / 5.0 %), year 2 +0.21 (SE 0.10, p = .04); middle schools 5,519 pupils, year 2 +0.10 (ns; treatment group 0.31 and 0.35 SD lower at pretest in the two cohorts, p = .02).
- Abstract (paraphrased): the second-year high-school effect is enough to move the median student up about eight percentile points.
Role for C5
Limits (weaken, 2026-10-08): qualifies the near-zero school-use bound from Steenbergen-Hu & Cooper 2013 — in routine adoption, gains were absent in the first year but about +0.2 SD for high schools in the second. Does not touch the near-human-tutor comparison in controlled experiments; 0.2 SD is far below it. A blended curriculum (software plus classroom lessons), not software alone.
Split (2026-10-08): this role now belongs to C10 (everyday school use), where the 2026-10-08 ledger entry moved.
Metadata as recorded
- ID
- AI-020
- Tier
- RCT
- Closed-book follow-up
- ja (end-of-year standardised algebra proficiency exam, not the software)
- Titel
- Effectiveness of Cognitive Tutor Algebra I at Scale
- Författare
- Pane, J. F.; Griffin, B. A.; McCaffrey, D. F.; Karam, R.
- År
- 2014 (Educational Evaluation and Policy Analysis 36(2), 127–144; RAND addendum WR-1050, May 2014)
- Källa/DOI
- https://doi.org/10.3102/0162373713507480
- URL
- https://eric.ed.gov/?id=EJ1024233 (abstract); addendum https://www.rand.org/pubs/working_papers/WR1050.html
- Typ
- cluster-randomised effectiveness trial: middle and high schools in seven US states matched in pairs and randomised to keep their algebra curriculum or adopt Cognitive Tutor Algebra I for two years, implemented as independent adopters would; standardised algebra proficiency posttest
- Åldersgrupp
- grundskola-hog;gymnasium
- Teman
- ai-och-larande;optimalt-larande-grunder
- Hämtad
- 2026-10-08
- Paywall
- Sage (closed); RAND addendum open
The original files (PDF or abstract) are kept in the project archive and are not published here.