Science & Technology (Commonwealth Union) – Are generative artificial intelligence tools like ChatGPT genuinely capable of creativity? A team of researchers led by Professor Karim Jerbi of the Université de Montréal’s Department of Psychology—working alongside AI pioneer Yoshua Bengio, also from the same university—has released the most extensive comparison to date between human creativity and that of large language models.
Published in Scientific Reports (Nature Portfolio), the study suggests that generative AI has crossed an important threshold. The results show that these systems can now outperform the average human in measures of creativity. That said, individuals with exceptionally high creative ability continue to outshine even the most advanced AI models.
To reach these conclusions, the researchers evaluated the creative output of multiple large language models, including ChatGPT, Claude and Gemini, and benchmarked them against responses from nearly 100,000 human participants. The findings indicate a clear shift: certain AI systems, notably GPT-4, now score higher than the human average in tests designed to measure divergent linguistic creativity.
“Our study shows that some AI systems based on large language models can now outperform average human creativity on well-defined tasks,” said Professor Karim Jerbi. “This result may be surprising — even unsettling — but our study also highlights an equally important observation: even the best AI systems still fall short of the levels reached by the most creative humans.”
Analyses led by the study’s two joint first authors — postdoctoral fellow Antoine Bellemare-Pépin of the Université de Montréal and doctoral researcher François Lespinasse from Université Concordia — point to a striking new insight. Although certain generative AI models now outperform the average human in creative tasks, the very highest expressions of creativity remain uniquely human.
Indeed, the mean scores of the top half of human participants exceed those achieved by every AI system evaluated, while the most creative 10 percent of individuals widen this advantage even further.
Professor Karim Jerbi, associate professor at Mila indicated that they designed a robust framework that enables a direct comparison between human and artificial creativity using identical assessment tools and this work draws on data from over 100,000 participants and was conducted in collaboration with Jay Olson from the University of Toronto.
Researchers of the study worked out a plan to measure AI with human creativity. To assess creativity across humans and AI alike, the researchers employed several complementary methods. Central among them was the Divergent Association Task (DAT), a psychological measure of divergent thinking — the capacity to produce a wide range of diverse and original ideas from a single prompt.
Designed by study co-author Jay Olson, the DAT requires participants—whether human or artificial—to come up with ten words that are as conceptually unrelated to each other as possible. For instance, someone demonstrating high creativity might offer a list such as: “galaxy, fork, freedom, algae, harmonica, quantum, nostalgia, velvet, hurricane, photosynthesis.”
Importantly, how people perform on this task closely aligns with their results on other well-established creativity assessments commonly used to evaluate idea generation, writing ability, and creative problem-solving. This shows that, despite relying on language, the DAT does more than test vocabulary knowledge. Instead, it taps into broader cognitive processes underlying creative thinking that extend well beyond linguistic skills. Another key benefit is its efficiency: the test takes just two to four minutes to complete and is readily available online to the public.
Building on this reasoning, the researchers next explored whether strong AI results on such a simple exercise—producing a small set of highly distinct words—would carry over to more demanding creative tasks that resemble real-world creative work. To do this, they directly compared AI systems with human participants on creative writing challenges, including composing haiku, drafting film plot summaries, and writing short stories. Once again, the most talented human creators maintained a clear edge, even though AI models were sometimes able to surpass the creativity levels of average human participants.
