Science & Technology, Canada (Commonwealth Union) – Researchers from Canada and the United Kingdom are raising a concern that could potentially hinder the advancement of artificially intelligent chatbots: the excessive output of these chatbots might eventually overwhelm the human-generated internet data that serves as their training material.
Generative AI models, including large language models like ChatGPT and creative tools such as Stable Diffusion, rely on vast amounts of internet data to form pattern-based predictions and gain an understanding of human text and images.
However, the landscape of the internet itself is on the brink of transformation due to the increasing prevalence of AI-generated content. This means that upcoming AI models will often be learning from the output of their predecessors.
This AI “ouroboros,” likened to a snake consuming its own tail, has the potential to create challenges for future generations of AI chatbots. It could disrupt their predictive abilities, as suggested by a pre-print paper authored by researchers from the University of Toronto, University of Oxford, University of Cambridge, University of Edinburgh, and Imperial College London.
While the paper has yet to undergo peer review, it outlines the possibility of an effect referred to as “model collapse.” This phenomenon could lead to issues where the quality of AI predictions deteriorates over time due to the recursive nature of learning from previous AI-generated data.
Nicolas Papernot, co-author of the paper and an assistant professor at the University of Toronto, draws a comparison to illustrate this concept pointing out that a good analogy for this is when photocopying of a piece of paper, and then it is photocopied the photocopied – we begin seeing more and more artifacts. Papernot is affiliated with both the Edward S. Rogers Sr. Department of Electrical and Computer Engineering in the Faculty of Applied Science & Engineering and the Department of Computer Science in the Faculty of Arts & Science at the University of Toronto.
“Eventually, if you repeat that process many, many times, you will lose most of what was contained in that original piece of paper.”
Papernot, an associate of the Schwartz Reisman Institute for Technology and Society, a strategic initiative at the University of Toronto, alongside his collaborators, devised simplified mathematical models to explore the potential dynamics of this deteriorative learning process.
Contemporary AI chatbots undergo training using data sourced from the internet, which is meticulously curated to encompass a broad spectrum of human information, ranging from the most common occurrences to the outliers and every nuance in between.
However, Papernot raises a concern that the rise of AI-generated content might “contaminate” the internet, altering the data pool to no longer mirror reality but rather the reality that large language models (LLMs) predict. As this contaminated data is fed into the subsequent generation of chatbots, their predictions could become skewed, emphasizing likely events excessively while underemphasizing rare scenarios. This imbalance could give rise to apprehensions regarding fairness and accuracy.
These errors become more pronounced with each successive iteration of the model; he explains. In the advanced stages of model collapse, the tainted data inherited from previous iterations accumulates and converges into a distorted portrayal of reality that bears little resemblance to our actual world. This renders the model’s predictions almost ineffective.
Papernot emphasizes that these findings raise uncertainty about the presumption that the current rapid progress in large language model (LLM) technology will continue without hindrance.
He points out that the essence of their paper shows that there is currently a fundamental challenge with the prevailing method of training models, which means heavy reliance on internet data may not be sustainable for further scaling of model training.
One proposed solution to address this challenge could involve training models to differentiate between content generated by humans and that produced by machines. However, Papernot notes that this approach might be complex, as technological advancements could blur the lines of distinction between the two sources of content.
