17/02/2026
I found this study through Matthew Connelly's opinion piece in the New York Times and thought it was worth revisiting.
Niloy et al. (2024) ran one of the first large experiments on ChatGPT and creative writing. "Is ChatGPT a Menace for Creative Writing Ability? An Experiment," published in the Journal of Computer Assisted Learning.
600 university students. Control group vs. experimental group with ChatGPT 3.5 access. Pre-test showed no difference between groups.
After using ChatGPT, the experimental group's creativity scores dropped significantly (p = 0.000). Accuracy and originality declined. "The generated content lost its originality compared to the state when ChatGPT did not intervene" (p. 926).
Two things did improve: elaboration and presentability. ChatGPT helped students expand ideas and structure content more clearly.
So the trade-off was specific. Clarity went up. Originality went down.
Now here's the thing. This was ChatGPT 3.5. Compared to GPT-5.2, 3.5 was primitive. No scaffolding was provided, no reflection, no pedagogical structure., and students were handed a tool and left alone with it.
We know from recent research that conditions matter. When students ask AI purposeful questions, writing improves (Cheng et al., 2025). When AI feedback is embedded in peer review and reflection, students evaluate it critically (Sperber et al., 2025).
The core insight still holds: passive AI use hurts creativity. That was true in 2024. It's true now. The tool has changed. The human tendency toward cognitive shortcuts hasn't.
Link in the first comment.
References
Cheng, Y., Fan, Y., Li, X., Chen, G., Gašević, D., & Swiecki, Z. (2025). Asking generative artificial intelligence the right questions improves writing performance. Computers and Education: Artificial Intelligence, 8, 100374.
Niloy, A. C., Akter, S., Sultana, N., Sultana, J., & Rahman, S. I. U. (2024). Is ChatGPT a menace for creative writing ability? An experiment. Journal of Computer Assisted Learning, 40(2), 919–930.
Sperber, L., MacArthur, M., Minnillo, S., Stillman, N., & Whithaus, C. (2025). Peer and AI Review + Reflection (PAIRR): A human-centered approach to formative assessment. Computers and Composition, 76, 102921.