Less than two years ago, the introduction of ChatGPT sparked a wave of excitement in the field of generative AI, with many experts heralding the advent of a fourth industrial revolution poised to reshape global industries. Enthusiasts predicted a seismic shift in the job market, with artificial intelligence taking center stage in various sectors. In March 2023, Goldman Sachs issued a stark forecast, predicting the loss or degradation of 300 million jobs due to AI advancements. The world braced for an unprecedented transformation.
However, 18 months later, the anticipated transformation remains largely unrealized. Contrary to initial expectations, generative AI has not revolutionized the business landscape. Numerous AI-driven projects have been shelved, highlighting the technology’s current limitations. One notable example is McDonald’s attempt to automate drive-through ordering. This project gained notoriety on TikTok after the AI system produced a series of humorous and erroneous orders. Similarly, government initiatives aimed at using AI to summarize public submissions and calculate welfare entitlements have encountered significant setbacks, leading to their cancellation.
This raises an important question: what went wrong?
Understanding the AI Hype Cycle
Generative AI, like many emerging technologies, appears to be following a trajectory known as the Gartner hype cycle, a model developed by the American technology research firm Gartner. This widely recognized model describes a recurring pattern where initial technological success leads to inflated public expectations, which eventually give way to disappointment as these expectations fail to materialize. The cycle begins with a “peak of inflated expectations,” followed by a “trough of disillusionment,” before progressing to a “slope of enlightenment” and ultimately reaching a “plateau of productivity.”
A Gartner report published in June 2023 indicated that most generative AI technologies were either at the peak of inflated expectations or still climbing toward it. The report suggested that these technologies are likely two to five years away from becoming fully productive.
While numerous compelling prototypes of generative AI products have been developed, translating these prototypes into practical applications has proven to be a significant challenge. A study published by the American think tank RAND last week revealed that 80% of AI projects fail, a failure rate more than double that of non-AI projects.
The Challenges Facing Generative AI
The RAND report highlighted several obstacles hindering the successful implementation of generative AI. Among these challenges are the substantial investment requirements for data and AI infrastructure, as well as a shortage of skilled human talent. However, the unique limitations of generative AI present an even greater hurdle.
For instance, generative AI systems have demonstrated the ability to solve complex tasks, such as university admission tests, yet they often struggle with simple, straightforward tasks. This disparity makes it difficult to accurately assess the potential of these technologies, leading to misplaced confidence in their capabilities.
One might assume that if an AI system can solve intricate differential equations or compose a well-written essay, it should easily handle basic tasks like taking drive-through orders. However, recent studies have shown that large language models, such as GPT-4, often fail to meet the expectations placed upon them. More alarmingly, these models tend to perform poorly in high-stakes situations where incorrect responses could have severe consequences.
These findings suggest that generative AI models can create a false sense of confidence among users. The models’ ability to answer questions fluently may lead to overly optimistic conclusions about their capabilities, resulting in their deployment in scenarios for which they are ill-suited.
Experience from successful AI projects underscores the difficulty of ensuring that a generative model consistently follows instructions. For example, the Khanmigo tutoring system developed by Khan Academy frequently disclosed correct answers to questions, despite being explicitly instructed not to do so.
In conclusion, while generative AI holds immense potential, its current limitations and challenges must be carefully navigated. The technology’s journey through the hype cycle serves as a reminder that while the promise of AI is great, realizing its full potential requires time, investment, and a clear understanding of its capabilities and limitations.
