Home · Open questions

Open questions

Each attack that does not settle things leaves a question behind. These are the ones nobody, including us, can answer yet.

25 open · 2 parked with the resting definition.

1. Question 1 · Retrieval beats restudy

   Twelve hours after practice, rereading did as well as recalling with the answers shown afterwards (Abel et al. 2019). Is that only true for short lists of unrelated word pairs, where each person tried both methods, or would rereading also keep up on the same day with real texts, or when different people use each method?

   Study detail

   (Touches C0. Replaces an earlier question, resolved 2026-10-05: no advantage at 12 hours with feedback, an advantage from about a day on, so C0's "later" now means about a day or more.)

   C0 · Retrieval beats restudyOnce learned, recalling repeatedly with feedback beats rereading on tests days later.6 of 14 attacks left it unchanged

2. Question 3 · Retrieval beats restudy

   In Roediger & Karpicke 2006, did the parts of the text that people failed to recall right after studying still benefit from testing, compared with rereading, on the later test?

   Study detail

   (Touches C0.)

   C0 · Retrieval beats restudyOnce learned, recalling repeatedly with feedback beats rereading on tests days later.6 of 14 attacks left it unchanged

3. Question 4 · Paper vs screen, deep reading

   With self-paced reading, is C3’s remaining paper advantage explained by medium preference — Ackerman & Goldsmith 2011 found screen inferiority under free study time among strongly paper-preferring students, while Delgado & Salmerón 2021 and Ackerman & Lauterman 2012 found equal scores — and does screen overconfidence persist when readers set their own pace (present in Ackerman & Lauterman 2012 Exp 2, absent in Exp 1 and in Delgado & Salmerón 2021)?

   Study detail

   (Replaces the self-paced question, resolved 2026-10-06: C3 weakened — the comprehension advantage is clearest under time limits and not established for self-paced reading.)

   C3 · Paper vs screen, deep readingPast early primary, paper often edges screens for timed deep reading.4 of 7 attacks left it unchanged

4. Question 6 · Spacing and interleaving

   Does spaced practice beat ordinary good revision, not just cramming?

   Study detail

   In a trial in 125 English schools, a spaced chemistry revision programme did no better on GCSE chemistry than the schools' usual revision (Hodgen, Bretscher, Hardman, Anders & Lawson 2023, SMART Spaces / EEF). That comparison is usual revision — not massed or blocked practice — so it does not kill C11, but it leaves open whether spacing still helps when schools already revise well.

   C11 · Spacing and interleavingSpaced, mixed practice beats cramming on tests weeks later; year-long gains unshown.2 of 4 attacks left it unchanged

5. Question 7 · Distracting extras

   Does the harm from distracting extras depend on the learning being mandatory, and does it hold when transfer is measured on its own?

   Study detail

   Wesenberg, Jansen, Krieglstein, Schneider & Rey 2025 found that extras lowered test scores when a lesson was declared mandatory but not when it was voluntary; the outcome was one combined recall-and-inference score, and separate transfer items were dropped for low reliability, so the study does not test C8's transfer claim.

   C8 · Distracting extrasUnmarked off-topic extras can hinder applying knowledge; the harm isn't consistent.3 of 5 attacks left it unchanged

6. Question 8 · Later school start

   Who gains most from a later school start: weaker students or better-off students?

   Study detail

   Edwards 2012 (all Wake County, North Carolina, middle schools, 1999–2006; natural variation in start times) found the test-score gain was stronger in the lower part of the score distribution (per the abstract). James et al. 2023 (START/LEARN, five Minneapolis–St. Paul high schools, a 50–65-minute delay) found larger grade gains for students without free or reduced-price lunch and with a college-graduate parent. The two studies differ in age, outcome (test scores vs grades) and what "weaker" means (low scorers vs lower family income), so neither narrows C14 on its own; whether a later start narrows or widens gaps is open.

   C14 · Later school startA later school start can add teen sleep; better results aren't guaranteed.2 of 3 attacks left it unchanged

7. Question 9 · Growth mindset for struggling students

   Does a growth-mindset lesson help struggling secondary students outside the US online format?

   Study detail

   In 202 public secondary schools in Salta, Argentina, a short paper-based version (read a text, write a letter to a classmate) had no effect on grade-12 maths or reading scores two months later, ruling out effects above .07 SD, and showed no difference for students who had repeated a grade (Ganimian 2020, Educational Evaluation and Policy Analysis). C13 names the online lesson, and the trial was not preregistered, so it does not kill C13; it leaves open whether any version of the lesson helps lower achievers outside the settings where it was developed.

   C13 · Growth mindset for struggling studentsA short growth-mindset lesson does little on average; extra help for struggling teens is unproven.0 of 1 attack left it unchanged

8. Question 10 · Retrieval beats restudy

   Does recalling beat rereading for learning maths procedures and problem solving, a day or more later, when practice comes with feedback?

   Study detail

   A preregistered meta-analysis of 7 maths studies found only a small, uncertain advantage for testing over restudy (g = 0.18, 95% CI −0.07 to 0.44; Murray, Horner & Göbel 2025), but only three of the seven gave feedback throughout (a fourth in one condition) and 42% of the effect sizes came from tests less than a day after practice, conditions C0 already leaves out; in one experiment with feedback and equal practice time, solving practice problems beat studying worked examples a week later (70% vs 41% of problems solved; Yeo & Fazio 2019, Experiment 2, 20 per group). (Held 2026-10-18 from the nightly attack run; not a verdict.)

   C0 · Retrieval beats restudyOnce learned, recalling repeatedly with feedback beats rereading on tests days later.6 of 14 attacks left it unchanged

9. Question 11 · AI deskilling

   When doctors who had been using AI to spot polyps did colonoscopies without it, did they find fewer adenomas because the AI had dulled their skill, or because something else changed at the same time?

   Study detail

   In four Polish centres, unaided detection fell from 28.4% to 22.4% in the three months after AI arrived (Budzyń et al. 2025), but the colonoscopies done with AI in those same months found adenomas in only 25.3%, also below the 28.4% before; the fall was largest in the centres that started highest (the authors raise regression to the mean), and the patient mix changed when Poland paused its screening programme. A prospective Norwegian study that followed the same endoscopists before, during and after AI use found no drop in unaided detection (Pedersen et al. 2026). C18 already says such a loss is not established; this asks what drove the Polish drop. (Added 2026-10-19 from the study-mining run; not a verdict.)

   C18 · AI deskillingRelying on AI at work may wear down the skill it replaces.0 of 1 attack left it unchanged

10. Question 12 · Distracting extras

    Do distracting extras mainly hurt learners with weaker academic prerequisites?

    Study detail

    In a lab experiment with 113 German university students (a lightning lesson, the same 6.5 minutes of study for everyone, a test right after), interesting but irrelevant pictures and captions did not significantly lower transfer overall (p = .160), but they did for the half of the students with weaker school-leaving grades (d = −0.60) and not significantly for the half with better grades (d = −0.21); prior knowledge and spatial ability made no difference (Bender, Renkl & Eitel 2021, Applied Cognitive Psychology). It is one study, a split at the median within a group of mostly good students, and only one of six moderator tests came out significant, so it does not narrow C8; it leaves open who bears the harm. (Held 2026-10-19 from the nightly attack run; not a verdict.)

    C8 · Distracting extrasUnmarked off-topic extras can hinder applying knowledge; the harm isn't consistent.3 of 5 attacks left it unchanged

11. Question 13 · Learning by teaching

    Does explaining what you learned to absent others in writing help on a later test, or only teaching aloud?

    Study detail

    In two experiments with Dutch students, explaining on a webcam video to imagined fellow students beat restudying, but typing the same explanation did not (d = 0.19, not significant; delayed-test means 22.17 against 21.53; Hoogerheide et al. 2016), and a cumulating meta-analysis of 18 studies found a benefit for spoken explaining (g = 0.34) but none for written explaining (g = −0.07, six comparisons, mostly tests right after; Lachner, Jacob & Hoogerheide 2021). Yet writing a word-for-word teaching script beat studying one's own notes on a test a week later (d = 0.53; Lim, Wong & Lim 2021). C15 does not say how the explaining is done; whether written teaching earns the delayed benefit is open. (Held 2026-10-20 from the nightly attack run; not a verdict.)

    C15 · Learning by teachingTeaching what you learned can help later, if you studied to teach.0 of 1 attack left it unchanged

12. Question 14 · Later school start

    Does starting school later still add sleep once school already starts after about 8:00?

    Study detail

    In a US national sample, each half hour of later start went with more sleep at schools starting at 8:00 or earlier, but not at schools starting later (Paksarian et al. 2015, as reported in the Cochrane review by Marx et al. 2017). Yet a Toronto school that moved from 9:00 to 10:00 and a Wellington school comparing 9:00 with 10:30 both reported more sleep with the later start (Brown 2011; Borlase 2013, as reported by Marx et al.). All three compare schools or year groups without random assignment, and the review says most studies did not look at how big the delay was or what time it moved to. C14 does not say how late is late enough; whether later starts help beyond about 8:00 is open. (Added 2026-10-21 from the study-mining run; not a verdict.)

    C14 · Later school startA later school start can add teen sleep; better results aren't guaranteed.2 of 3 attacks left it unchanged

13. Question 15 · Distracting extras

    Does a learner's state of arousal decide whether distracting extras do harm?

    Study detail

    In a lab experiment with 100 German university students (a self-paced web lesson on sleep, about 15 minutes, tests right after), interesting but irrelevant text passages lowered transfer when learners heard a looped heartbeat at 70 beats per minute (20.24 against 23.72 of 32 points) but not when it ran at 90, which raised self-reported energy (24.44 against 23.96; interaction F(1,96) = 15.66, p < .001; Schneider, Wirzberger & Rey 2019, Applied Cognitive Psychology). It is one study with 25 learners per group, an unusual way of raising arousal, no replication, and some printed statistics that do not add up, so it does not narrow C8; it leaves open whether the harm depends on the learner's state as well as on the material. (Held 2026-10-23 from the nightly attack run; not a verdict.)

    C8 · Distracting extrasUnmarked off-topic extras can hinder applying knowledge; the harm isn't consistent.3 of 5 attacks left it unchanged

14. Question 16 · Systematic phonics

    Against reading teaching that already covers letter–sound links now and then (as "balanced" approaches and Reading Recovery do), how much does a planned phonics sequence add to at-risk beginners' decoding?

    Study detail

    Re-analyses of the National Reading Panel's studies that sorted the comparison groups by how much phonics they got found that systematic phonics beat "some" phonics by about d = 0.24 or 0.19 depending on the weighting (Camilli, Vargas & Yurecko 2003), and by 0.12, not significant, in a later multilevel model (Camilli, Wolfe & Smith 2006, as reported by the authors in 2008; a rival re-estimate on the same coding gave 0.18, p = .04, Stuebing et al. 2008); in one small study of at-risk first graders, adding systematic phonics to Reading Recovery left word reading and nonword reading about equal (d = −0.12 for each). These figures pool all reading outcomes and all readers, so they do not narrow C12's decoding claim, and one randomised head-to-head trial with at-risk first graders favours the planned sequence on decoding new words (Mathes et al. 2005). In that trial the comparison classrooms already taught letter–sound links in 72.72% and sounding out in 82.36% of 90 observed lessons; against them the planned-sequence group gained 11.00 W-score points more on decoding new words (effect size 0.76) and the responsive group 4.11 (not significant), but both groups also got 40 extra minutes of small-group teaching a day, so the planned sequence and the extra time cannot be told apart in that comparison (full text read); how large the advantage is over incidental phonics is open. (Held 2026-10-24 from the nightly attack run; not a verdict. Since the same day C12 claims only an edge over little or no phonics, and this question holds the edge over balanced or incidental phonics. Mathes classroom detail added 2026-11-11 from study mining.)

    C12 · Systematic phonicsTeaching letter–sound links step by step helps struggling beginners decode; comprehension gains unshown.1 of 2 attacks left it unchanged

15. Question 17 · Sleep before learning

    Does spacing study across days protect learning from a week of short sleep, or were the spaced words simply over-learned?

    Study detail

    In a boarding-school trial with 56 Singapore teenagers (15–19), a week of 5-hour nights lowered later recall of English vocabulary crammed in single sessions but not of words spread over four days, with the same total study time (spacing × sleep interaction, P < 0.01; Huang et al. 2016, SLEEP). The authors say recall of the spaced words was very high in both groups, so a cost may have been hidden by a ceiling; there was no power analysis, and short sleep came both before and after learning. It is one study, so it does not narrow C21 or C11. (Added 2026-10-28 from study mining; not a verdict.)

    C11 · Spacing and interleavingSpaced, mixed practice beats cramming on tests weeks later; year-long gains unshown.2 of 4 attacks left it unchangedC21 · Sleep before learningEnough sleep before learning protects what was learned.1 of 3 attacks left it unchanged

16. Question 18 · Background knowledge

    Do weak readers who know a topic well really understand a text on it as well as strong readers who know little, once the text is an ordinary school text rather than a baseball play-by-play?

    Study detail

    The best-known study behind this idea (Recht & Leslie 1988, seventh and eighth graders) compared groups that also differed by sex (the strong-reader, low-knowledge group had 3 boys and 13 girls; the weak-reader, high-knowledge group 12 boys and 4 girls), used a knowledge test that was mostly baseball vocabulary, with only 15 of its 39 questions touching the passage, and a text with almost no words linking one event to the next (Reynolds, Hattan & Markham 2024, Reading Research Quarterly, a review of 19 such "baseball studies" from 1978 to 2018). The review reports no result against C17, and it points to studies finding that clearer, better-connected texts help readers with little knowledge most. The first part of C17 rests on this kind of study, so whether it holds for school texts and topics is open. (Added 2026-10-30 from study mining; not a verdict.)

    C17 · Background knowledgeKnowing a topic helps you understand texts on it; teaching it doesn't always lift scores.1 of 5 attacks left it unchanged

17. Question 19 · Tangible rewards and interest

    Does the drop in interest after a reward is withdrawn last, or fade within weeks?

    Study detail

    In a re-analysis of the reward experiments, only 2 of 7 studies that measured free-time play again a few weeks later still found a significant drop (Cameron, Banko & Pierce 2001, The Behavior Analyst), while the competing meta-analysis found drops in children right away, within a week and after a week (Deci, Koestner & Ryan 1999, as reported by Cameron and colleagues). A large preregistered online study with adults measured free choice only minutes later (a drop of about 0.2 standard deviations after pay per correct answer was removed; Bitter et al. 2022, Technology, Mind, and Behavior). C22 does not say how long the drop lasts; that is open. (Added 2026-10-30 from study mining; not a verdict.)

    C22 · Tangible rewards and interestPromised prizes for interesting work can cut later free choice.1 of 1 attack left it unchanged

18. Question 20 · Spaced L2 practice

    For second-language grammar, do longer gaps between practice sessions really beat shorter ones on tests a month later?

    Study detail

    C23 says longer gaps win on delayed tests, resting on a meta-analysis read only in its abstract (Kim & Webb 2022). In a randomised study with 60 Japanese university students learning the verb endings of a made-up language, four sessions 3.3 days apart beat sessions 7 days apart on tests 7 and 28 days later (d about 0.5; Suzuki 2017, Language Learning), and in a smaller one with 40 US learners of Japanese, sessions 1 day apart did as well as 7 days apart a month later (Suzuki & DeKeyser 2017, Language Teaching Research). Both compared two spaced schedules, so C23's main spaced-versus-massed claim is untouched; whether the longer-gap advantage holds for grammar, or mainly for vocabulary, is open until the meta-analysis's full text (its moderators) can be read. (Held 2026-10-30 from the nightly attack run; not a verdict.) Full text read 2026-11-05: in Kim & Webb 2022, longer beat shorter spacing on delayed grammar tests (g = 0.56, 95% CI 0.06 to 1.06; 8 comparisons), but across all targets longer gaps between separate sessions did not reliably beat shorter ones (g = 0.18; 21 comparisons) and the paper does not split grammar by within- versus between-session spacing, so the question stays open; C23 was narrowed the same night.

    C23 · Spaced L2 practiceSpreading L2 vocabulary practice beats massing it on delayed tests; grammar unproven.0 of 1 attack left it unchanged

19. Question 21 · Explicit L2 teaching

    Does teaching that states the second-language rule still beat implicit teaching on tests given weeks later?

    Study detail

    C24 says explicit teaching gives larger gains, resting on two meta-analyses read only in their abstracts (Norris & Ortega 2000; Spada & Tomita 2010). A later meta-analysis of 54 studies from the same six journals reports only "a minor difference" between explicit and implicit instruction in its abstract (Kang, Sok & Han 2019, Language Teaching Research 23(4), 428–453, DOI 10.1177/1362168818776671), and secondary summaries of its tables say implicit instruction did better on delayed tests (about g = 1.76 against 0.77, 11 studies each). Its full text is closed and was not read, so this is a lead, not a verdict; whether the explicit advantage lasts is open. (Added 2026-10-31 from the nightly attack search; not a verdict.)

    C24 · Explicit L2 teachingStating the rule beats leaving it implicit for learning L2 grammar forms.not tested yet

20. Question 22 · Homework and achievement

    Is homework's link with achievement really stronger in secondary school than in the early grades?

    Study detail

    C27 says it is, following Cooper, Robinson & Patall 2006 (US studies). A later maths-and-science meta-analysis is reported, in a German research clearinghouse review, to have found the link stronger in primary school (r = 0.32) and upper secondary (r = 0.30) than in lower secondary (r = 0.15) (Fan, Xu, Cai, He & Fan 2017, Educational Research Review 20, 35–54, DOI 10.1016/j.edurev.2016.11.003). The paper itself is closed and was not read, so this is a lead, not a verdict. (Added 2026-10-31 from the nightly attack search; not a verdict.)

    C27 · Homework and achievementHomework goes with higher results; the link is clearly stronger in secondary school, but experiments, mostly in grades 3–6, also show gains.0 of 1 attack left it unchanged

21. Question 23 · Learning outside the classroom

    Do pupils who are taught outdoors every week, as in Scandinavian "udeskole", learn more than pupils taught the same content indoors?

    Study detail

    C28 says curriculum-linked learning outside the classroom raises achievement by about 0.3 standard deviations in randomised studies, resting on a meta-analysis of out-of-school learning in general (Delen et al. 2026, full text read 2026-11-08), in which the four Scandinavian studies averaged g = 0.18 with an interval crossing zero. A systematic review of regular outdoor teaching (at least four hours a week for at least two months) found 13 studies up to 2016, mostly case studies and none randomised; only one compared standardised achievement tests with a control group (90 US fifth graders, higher reading and writing scores, p = 0.03, no effect size, maths not reported), and none tested again later (Becker et al. 2017, International Journal of Environmental Research and Public Health). A later Danish quasi-experiment (23 outdoor-maths classes against 12 comparison classes, ages 9–13, one school year) reports no difference in maths in its abstract (Otte et al. 2019, Educational Research; full text not yet read). Whether regular outdoor teaching, as opposed to single trips, raises achievement is open. (Added 2026-11-02 from study mining; not a verdict.)

    C28 · Learning outside the classroomCurriculum-linked field trips and outdoor lessons beat classroom-only teaching by about 0.3 SD in randomised studies (0.5 across all studies).0 of 1 attack left it unchanged

22. Question 24 · Task feedback vs praise

    Does showing a grade next to written comments take away the benefit of the comments?

    Study detail

    C4 compares comments with a grade or praise on its own; it does not say what happens when both are given. In a classroom experiment in 12 grade 5–6 classes (132 pupils measured), grades plus comments undermined later interest and performance about as much as grades alone, while comments alone helped (Butler 1988, British Journal of Educational Psychology; abstract read, full text not yet read). In an experiment with 464 US first-year university students revising a course-exam essay a week after the first draft, detailed comments raised the revised score strongly whichever way they were framed, and students who also saw their draft grade scored lower when the comments came from the instructor (adjusted means 82.74 without the grade, 79.63 with it), but not when they came from a computer; the overall drop from seeing a grade was small (F(1, 450) = 4.07, which does not pass the authors' own corrected significance level of .0083), and there was no later task (Lipnevich & Smith 2009, Journal of Experimental Psychology: Applied). In four intact grade 9–10 history classes in one Madrid school (99 pupils; about 92 in the performance analysis), where every task got brief written comments and the grade was shown for two tasks and withheld for two, the class that started seeing grades scored lower on the next task than the class that had just stopped seeing them (5.40 against 6.80 out of 10, blind re-graded), but on the following task the two were level again (6.86 against 6.73); comments were short and mostly corrections, there was no comments-only or grade-only class, and only two classes per condition (Panadero & Sánchez-Iglesias 2025, Educational Assessment; full text read). Whether a grade alongside comments reduces later learning, rather than effort on one revision or one task, is open. (Added 2026-11-09 from study mining; not a verdict. Panadero & Sánchez-Iglesias added 2026-11-11 from study mining.)

    C4 · Task feedback vs praiseOn open-ended work, comments on how to improve beat a bare grade.1 of 2 attacks left it unchanged

23. Question 25 · Systematic phonics

    Does the decoding edge that systematic phonics gives at-risk beginning readers last after the teaching ends?

    Study detail

    C12's evidence is mostly measured right after teaching. In a meta-analysis of 71 reading-intervention groups that were tested again on average 11 months later, phonics interventions lost more of their effect than other kinds: weighted d = 0.29 at the end of teaching and 0.07 at follow-up (reading-skill outcomes 0.26 and 0.07; unweighted 0.44 and 0.25), while phonemic-awareness and comprehension interventions held up. But the weighted figure leans on two large classroom studies of mostly typical readers (n = 1,405, follow-up d = −0.13; n = 1,021, 0.12), the comparison groups had ordinary teaching that may itself include phonics, "phonics" meant any intervention with a phonics component, and 14 of the 16 phonics groups of at-risk, low or disabled readers still had positive follow-up effects (Suggate 2016, Journal of Learning Disabilities; full text read). Whether at-risk beginners' decoding edge lasts a year, against teaching with little or no phonics, is open. (Added 2026-11-09 from the nightly attack run; held block, not a verdict.)

    C12 · Systematic phonicsTeaching letter–sound links step by step helps struggling beginners decode; comprehension gains unshown.1 of 2 attacks left it unchanged

24. Question 26 · Spaced L2 practice

    Does spreading out second-language vocabulary practice help low-proficiency learners on later tests, or mainly learners who already know more of the language?

    Study detail

    C23 says spaced practice gives clearly better delayed results for vocabulary, resting on a meta-analysis that could not test proficiency because the studies reported it poorly (Kim & Webb 2022). In a classroom study with 199 Iranian young-adult learners of English, the same 100 minutes of idiom practice spread over five days beat practice massed into two sessions about 100 days later for mid- and high-proficiency learners (form recall 82.6 against 73.9 % and 87.9 against 78.9 %), but not for low-proficiency learners (67.4 against 71.5 %, no reliable difference), who did better with massed practice right after learning (Heidari, Siyanova-Chanturia & Elgort 2026, Language Teaching Research; full text read). The classes were intact, not randomised, the items were opaque idioms only, the paper's printed effect sizes do not match its own odds ratios, and the same first author's collocation study reports the opposite pattern (low-proficiency learners gained most; abstract only). Whether proficiency decides who gains from spacing is open. (Added 2026-11-10 from the nightly attack run; held block, not a verdict.)

    C23 · Spaced L2 practiceSpreading L2 vocabulary practice beats massing it on delayed tests; grammar unproven.0 of 1 attack left it unchanged

25. Question 27 · Phones in lessons

    Does a stricter phone rule (phones kept out of school, switched off all day, or handed in) help more than a looser one (phones kept on silent, or off only during lessons)?

    Study detail

    C6 and C36 treat bans and restrictions together. In English secondary schools, looser rules raised age-16 exam results more than strict bans, and for the pupils with the lowest results at age 11 the gain was nearly three times as large (0.305 against 0.109 standard deviations); the authors call the reason "unclear" (Beland & Murphy 2016, Labour Economics; full text read). In Norwegian middle schools, girls' GPA and externally marked maths rose more where phones were handed in or not allowed (GPA +0.12, maths +0.25 standard deviations four years after the ban) than where they were only silenced in lessons (Abrahamsson 2024, NHH discussion paper; full text read). Both studies date bans from headteachers' later recollections, define "strict" a little differently, and neither randomised the type of rule; a large US study of schools that locked phones in pouches for the whole day found test-score effects close to zero (Allcott et al. 2026, working paper). Whether the type of rule matters for learning, and for whom, is open. (Added 2026-11-13 from study mining; not a verdict.)

    C6 · Phones in lessonsPhone bans raised results in some places; no clear average gain.3 of 5 attacks left it unchangedC36 · Phone bans and lower achieversSchool phone bans raise test results more for lower-achieving pupils than for higher-achieving pupils.not tested yet

See how they fit on the map →