Making AI Tutoring Productive: Evidence from a Mastery-Based Math Practice Experiment
Added to the Kunskapsbank on 2026-09-07 but not on the old published site.
Lower secondary (grades 7–9) Across ages AI and learning
Links to claims
Supports (1)
| Claim | Basis |
|---|---|
| K25 (candidate) | Note carries the bank kill sentence behind K25 (group K25); no change in review |
Limits (1)
| Claim | Basis |
|---|---|
| C2 | Guard-railed AI inside a mastery rule gave a modest delayed gain over the same platform without AI; not answer-giving, so outside C2. |
Review state: first-pass draft, pending human review.
Source note (Swedish, verbatim from the Kunskapsbank)
Syntes: Oreopoulos et al. (2026) — NUMI mastery × AI tutor (AI-014)
Källa: Oreopoulos, P., Liut, M., Sungu, A., & Low, N. (2026). Making AI Tutoring Productive… NBER WP 35621 / EdWorkingPaper 26-1552. https://doi.org/10.3386/w35621
Original: 01-originaldokument/oreopoulos-etal-2026-numi-mastery-ai/ (full OA PDF)
ID: AI-014
Tier: RCT
Closed-book follow-up: ja (delayed test ~1 vecka; practiced vs unpracticed)
HE vs K–12: grundskola-hog (åk 6–8, >6 000 elever, Tennessee)
Evidensstyrka: hög design (individuell randomisering 2×2×2); modest/imprecise delayed effects; WP.
Metadata
- ID:
AI-014 - Tier: RCT
- N / design: fält-RCT (>6 000 middle-school; NUMI guard-railed LLM tutor × mastery)
- Population: grundskola-hog (grades 6–8)
- Closed-book follow-up: ja (delayed assessment ~1 vecka utan AI; practiced vs unpracticed)
- HE vs K–12: grundskola-hög / middle school (åk 6–8, Tennessee)
- Subject: Making AI Tutoring Productive: Evidence from a Mastery-Based Math Practice Experiment
Finding (+ comparator)
Comparator: samma CAL-plattform utan AI (± mastery-regel). AI → långsammare progression, färre försök, högre accuracy | attempt; bättre post-mistake recovery. Mastery ensam ↑ three-correct-in-a-row men inte delayed learning. Starkast delayed-signal: AI i mastery på practiced Exercise 1 ≈ +3 percentage points (p≈0,065).
Mechanism
Guard-railed LLM (håller inne facit) + mastery som skapar demand för hjälp efter fel → productive struggle i stället för crutch. Tidskostnad real: mer klocktid per återhämtning.
Limits
En session + 1-veckas delayed test (4 items); Exercise 2 kontaminerad av exposure; WP ej peer-review; USA middle school.
This claim dies if …
Replikerad K–12-RCT där guard-railed LLM utan mastery slår CAL på delayed closed-book eller där mastery+AI inte längre skiljer sig från CAL på practiced retention.
Swedish implication
Stop “ChatGPT under övning” utan struktur; start AI-tutor endast med (a) no-answer policy, (b) mastery/retrieval-krav, (c) delayed closed-book; measure practiced vs unpracticed efter ≥1 vecka (mall från NUMI/Bastani).
Links
- Relaterade: AI-003, AI-011, AI-015, AI-008, LEARN-001
- DOI: https://doi.org/10.3386/w35621
Metadata as recorded
- ID
- AI-014
- Tier
- RCT
- Closed-book follow-up
- ja (delayed assessment ~1 vecka utan AI; practiced vs unpracticed)
- HE vs K–12
- grundskola-hög / middle school (åk 6–8, Tennessee)
- Titel
- Making AI Tutoring Productive: Evidence from a Mastery-Based Math Practice Experiment
- Författare
- Oreopoulos, P.; Liut, M.; Sungu, A.; Low, N.
- År
- 2026
- Källa/DOI
- NBER WP 35621; EdWorkingPaper 26-1552. https://doi.org/10.3386/w35621 ; https://doi.org/10.26300/01qv-6c22
- URL
- https://edworkingpapers.com/sites/default/files/ai26-1552.pdf
- Typ
- fält-RCT (>6 000 middle-school; NUMI guard-railed LLM tutor × mastery)
- Åldersgrupp
- grundskola-hog (grades 6–8)
- Teman
- ai-och-larande; uppmarksamhet-och-minne; optimalt-larande-grunder
- Sparade fil
- Oreopoulos-etal-2026-NUMI-EdWorkingPaper.pdf (full OA)
- Hämtad
- 2026-09-07
- Paywall
- nej (EdWorkingPaper OA)
- Obs
- Delayed-test ≈+3 pp på practiced Exercise 1 under mastery+AI (p≈0,065); post-mistake recovery starkare. Preprint/WP.
Original files held in the project (not published): ABSTRACT.md, Oreopoulos-etal-2026-NUMI-EdWorkingPaper.pdf