Coach not crutch: Evidence that AI can improve writing skill despite reducing effort
Adults / lifelong learning AI and learning Reading and writing
Links to claims
Limits (1)
| Claim | Basis |
|---|---|
| C2 | AI tool handed over a finished rewritten cover letter during practice; unaided writing later better than after practice without AI (d = .38; one day later d = .41). Scopes C2 away from practice where the AI output is a worked example of the target skill; weakened 2026-10-05 (arXiv preprint). |
Review state: first-pass draft, pending human review.
Source note
Syntes: Lira et al. (2025, arXiv) — practising with an AI that hands over a finished rewrite improved unaided writing
Källa: Lira, B., Rogers, T., Goldstein, D. G., Ungar, L., & Duckworth, A. L. (2025). Coach not crutch: Evidence that AI can improve writing skill despite reducing effort. arXiv:2502.02880 (v4, 18 Feb 2026). https://doi.org/10.48550/arXiv.2502.02880 — preprint, not peer reviewed.
Verified numbers (full arXiv v4 text read 2026-10-05)
- Paradigm: pretest (rewrite a poor cover letter), short lesson on five writing principles, randomised practice, then a test rewriting a new cover letter without AI (copy-paste disabled); follow-up test one day later. The AI tool "instantly revised text based on these principles" — it hands the learner a finished letter.
- Study 2 (N = 2,238): practice quality with AI far higher (d = 1.01). Unaided test: practised-with-AI > practised-without-AI (d = .38, p < .001) and > no practice (d = .47). One day later: still > practised-without-AI (d = .41, p < .001) and > no practice (d = .46); practising without AI was no better than no practice (d = .05). Effort was lower with AI (about a quarter of the keystrokes; self-reported effort d = −.31). 95% made at least one edit to the AI output.
- Editors / Google study (N = 2,997): unaided test after practising with AI > Google search (d = .46) and > personalised feedback from professional editors (d = .20).
- Example study (N = 2,003): practice with AI d = 1.22; unaided test d = .34 vs practice without AI; merely seeing an AI-revised letter (no practice) helped as much (d = .03 vs practising with AI); at 24 h AI practice d = .29 and AI example d = .32 vs practice without AI.
- Authors' own limit: in other domains "merely observing a solution may be less informative" — the final answer to a maths or programming problem does not reveal the procedure.
Role for C2
Narrows C2: when the AI's finished output is itself a visible worked example of the skill being practised (here, a rewrite following the taught writing principles), outsourcing the draft during practice raised unaided skill immediately and one day later instead of lowering it. Does not touch the maths, programming and informational-text results that carry C2 (Bastani, Strömberg, Kreijkes, Bergh, Barcaui). Adults online, writing-style skill, single practice session, preprint. C2 weakened 2026-10-05.
Metadata as recorded
- ID
- AI-018
- Tier
- RCT
- Closed-book follow-up
- ja (writing test without AI, copy-paste disabled, immediately and one day later)
- Titel
- Coach not crutch: Evidence that AI can improve writing skill despite reducing effort
- Författare
- Lira, B.; Rogers, T.; Goldstein, D. G.; Ungar, L.; Duckworth, A. L.
- År
- 2025 (arXiv preprint 2502.02880, v1 Feb 2025; v4 18 Feb 2026; submitted to Nature Communications; not peer reviewed as of 2026-10-05)
- Källa/DOI
- https://doi.org/10.48550/arXiv.2502.02880
- URL
- https://arxiv.org/abs/2502.02880
- Typ
- preregistered online random-assignment experiments (Prolific adults); practice rewriting a cover letter with an AI tool that instantly produces a revised letter vs practice without AI vs no practice / editor feedback / Google search / AI example only; outcome = writing quality on a new cover letter without AI (GPT-4o rating, validated against human raters r = .70, plus pairwise hiring judgments)
- Åldersgrupp
- vuxen-livslangt (Prolific adults, M ≈ 36–42 years; most with college degrees)
- Teman
- ai-och-larande;lasning-och-skrivande
- Hämtad
- 2026-10-05
- Paywall
- OA (arXiv)
The original files (PDF or abstract) are kept in the project archive and are not published here.