For years, a silent, physical problem hid deep inside the hardware of artificial intelligence and not through any lack of ideas or data. So far, the bigger and more powerful an AI model is, the faster it needs to shuttle data between memory and processors over chips, which can create a choke point in the transfer of information that wastes time, energy and potential.
Now, how about we finally welcome the coming 3D chip? This new breakthrough promises faster processing, reduced power consumption, and a quantum leap in AI performance. It hints at a future where AI just might think quicker, scale smarter and no longer be encumbered by the confines of flat silicon.
Moore’s Law, where computing power was doubled every two years through the addition of more transistors on a microchip, was a brilliant innovation of a bygone era of computing. Today’s AI requires so many parameters that this approach has hit a speed bump, and the end result is a bottleneck that limits the power of AI in all of its forms.
Scientists have announced a game-changing new type of computer chip created by a team of researchers from Stanford University. This is more than just a speed bump solution, as there is a radical rethink of silicon layout on display here which involves stacking layers of compute and memory vertically on a single slice of silicon rather than placing it out flat. Initial tests have shown performance boosts of several multiples over existing technology, and simulation-based estimates have shown even greater performance benefits for future generations.
The issue for which this new semiconductor holds the key is the memory wall that engineers face, as in present-day chips, “the processors where all your logic runs and your memory where your data is stored are usually separate entities on a chip,” says Brad Peterson. As a result, data transfers between the processor and memory occur many times per second, and AI models become increasingly complex as the volume of data grows, leading to significant inefficiencies.
What’s revolutionary about the monolithic 3D chip is that it’s essentially bridging this entire gap instead of building layers of memory and processing elements on top of each other. A monolithic process builds all of these layers on top of each other in a single continuous process. This approach builds each level of the stack on the same wafer, allowing for a very high level of interconnects between these layers.
The differences are dramatic, as the hardware evaluations performed to date have provided a performance level of approximately four times that of similar two-dimensional processors, while simulations of added layers stacked on top appear to provide possibilities for a dramatic tenfold improvement or more in terms of practical AI processing. Certain simulations now indicate that energy performance levels could be improved in terms of balance between speed and power consumption by factors of 100 to 1,000 compared to traditional processors. Such a dramatic combination of speed gains and energy performance could allow a reduction in electricity bills related to AI processing at many data centres.
Why exactly is vertical stacking such a breakthrough? The answer lies in the nature of information transfer on the chip. On a two-dimensional board, signals must be passed laterally over a certain distance, and this takes time and power. However, the 3D chip creates hundreds or thousands of pathways, and these pathways are vertically connecting the layers. This technique creates a kind of high-speed highway inside the chip because it reduces the traffic jams that occur in the two-dimensional board.
Another area of immense advantage is power efficiency. The current generation of artificial intelligence computers consumes enormous amounts of power, which forces tech giants and data centres to spend millions of dollars on cooling and powering these systems; however, the new 3D architecture significantly reduces power consumption by minimising the distance that data travels and eliminating redundant data movements. This means more computing per watt of power consumption.
It is a big deal to see that a monolithic chip based on 3D technology has been developed in a foundry, which is a key stepping stone to industry change if technology wants to achieve it. AI is about to experience its most significant advancements yet, as the competition to build faster, more intelligent, and more efficient processors has literally reached new heights.
