Home · Sources · AI-016

Effects of LLM use and note-taking on reading comprehension and memory: A randomised experiment in secondary schools

Authors: Kreijkes, P.; Kewenig, V.; Kuvalja, M.; Lee, M.; Hofman, J. M.; Vitello, S.; Sellen, A.; Rintel, S.; Goldstein, D. G.; Rothschild, D.; Tankelevitch, L.; Oates, T. · Year: 2026 · Tier: RCT · DOI/URL: https://doi.org/10.1016/j.compedu.2025.105514

Upper secondary AI and learning Reading and writing Attention and memory

Links to claims

Supports (1)

Claim	Basis
C2	Secondary RCT: LLM-only study → lower 3-day retention/comprehension vs note-taking; learning measured without the tool; kill-condition test survived 2026-09-29.

Review state: first-pass draft, pending human review.

Source note (Swedish, verbatim from the Kunskapsbank)

Syntes: Kreijkes et al. (2026) — LLM vs note-taking in secondary schools

Källa: Kreijkes, P., Kewenig, V., Kuvalja, M., Lee, M., Hofman, J. M., Vitello, S., Sellen, A., Rintel, S., Goldstein, D. G., Rothschild, D., Tankelevitch, L., & Oates, T. (2026). Effects of LLM use and note-taking on reading comprehension and memory: A randomised experiment in secondary schools. Computers & Education, 243, 105514. https://doi.org/10.1016/j.compedu.2025.105514

Metadata

ID: AI-016
Tier: RCT
N / design: pre-registered; 405 Year 10 students, 344 analysed; LLM-only (GPT-3.5 turbo) vs Notes vs LLM+Notes; delayed tests 3 days later without the tool
Population: gymnasium (England, 14–15)
Subject: naturalistic LLM study aid for comprehension-focused reading vs traditional note-taking

Finding

Notes > LLM-only on literal retention (β=1.92, d=0.44), comprehension (β=0.95, d=0.38), and free recall (β=1.02, d=0.21).
LLM+Notes > LLM-only on literal retention (d=0.13) and comprehension (d=0.14); free recall ns.
Students preferred the LLM and rated it more helpful, while investing less time and effort; ~26% in LLM+Notes copied heavily (>70% trigram overlap) from LLM output into notes.

Mechanism

LLM-only study offloads effortful encoding; learning measured without the tool is lower than note-taking. Preference/helpfulness judgments misalign with retention.

Limits

No reading-only control; GPT-3.5; single short learning session; free-recall contrast underpowered relative to retention/comprehension; LLM was a custom chatbot with a comprehension meta-prompt (explanations/summaries), not a pure homework-answer bot.

This claim dies if …

A replicated secondary/HE study measuring without the tool finds typical GenAI/LLM study use during practice does not lower retention vs a non-AI control.

Swedish implication

Measure learning without the tool after LLM-supported reading; prefer note-taking (alone or carefully combined) over LLM-only for durable comprehension.

Metadata as recorded

ID: AI-016
Tier: RCT
Closed-book follow-up: ja (comprehension, literal retention, free recall 3 days later, no LLM)
Titel: Effects of LLM use and note-taking on reading comprehension and memory: A randomised experiment in secondary schools
Författare: Kreijkes, P.; Kewenig, V.; Kuvalja, M.; Lee, M.; Hofman, J. M.; Vitello, S.; Sellen, A.; Rintel, S.; Goldstein, D. G.; Rothschild, D.; Tankelevitch, L.; Oates, T.
År: 2026 (Computers & Education 243, 105514; available online 24 Nov 2025)
Källa/DOI: https://doi.org/10.1016/j.compedu.2025.105514
URL: https://www.sciencedirect.com/science/article/pii/S0360131525002829
URL (OA PDF mirror): https://researchdmr.com/files/LLM_Notes_Reading_Comp.pdf
Typ: pre-registered RCT with within- and between-participant elements (405 Year 10; 344 analysed)
Åldersgrupp: gymnasium (England, ages 14–15)
Teman: ai-och-larande; lasning-och-skrivande; uppmarksamhet-och-minne
Sparade fil: Kreijkes-etal-2026-LLM-note-taking.pdf
Hämtad: 2026-09-29
Paywall: Elsevier; OA PDF mirror fetched

Original files held in the project (not published): Kreijkes-etal-2026-LLM-note-taking.pdf