It is as big as a tablet, but it is not one of them; it’s a monster deep learning chip with over 1.2 trillion transistors
Deep learning algorithms don’t quite get along with general-purpose processors. its high intrinsic parallelism it fits much better with the architecture of the graphics processors, which has caused that many data centers and research laboratories dedicated to artificial intelligence have entrusted the processing effort to a more or less ambitious GPU cluster.
However, this is not the only option. It is also possible to deploy an infrastructure of chips linked by high performance connections and specifically designed to deal with the intrinsic high parallelism of deep learning algorithms. One of the companies that has solutions of this type is Intel. Its Loihi neuromorphic chip is manufactured with 14 nm photolithography and incorporates 128 cores and slightly more than 130,000 artificial neurons.
The Wafer Scale Engine chip developed by Cerebra incorporates 1.2 trillion transistors. And they are 1.2 billion of us, not of the Anglo-Saxons
IBM also has its own neuromorphic processor, a chip that its creators have called TrueNorth. It integrates 4096 cores, so that it is possible to connect several of them in a network with the purpose of emulating, according to IBM, a system with a million neurons and 256 million synapses. Intel, IBM and NVIDIA are some of the big companies that are involved in the development of hardware designed specifically for artificial intelligence, but they are not the only ones that have a say in this area.
And it is that the Californian company Cerebras has developed a chip designed specifically for deep learning. The funny thing is that it looks relatively little like a conventional processor. It doesn’t even look like the Intel or IBM hardware that I’ve briefly talked about in the previous paragraphs. As you can see in the cover photo of this article, it is much larger than a traditional chip. In fact, it seems likely that they are employing a full wafer to produce each of them. However, this is by no means the only characteristic for which the Cerebras chip is so rare.
One big chip is better than many small ones, according to Cerebras
Here’s another amazing tidbit about the Wafer Scale Engine processor (WSE), which is what its creators call it: it integrates nothing less than 1.2 trillion transistors. And that’s 1.2 trillion of us, not Anglo-Saxons, so this number equates to a monstrous 1,200,000,000,000 transistors. It’s certainly a figure that’s hard not to be surprised by despite the sheer numbers of transistors found in chips we’re all familiar with, like the CPUs and GPUs in our computers.
This huge number of transistors responds to the approach that the engineers who have designed the WSE chip have chosen, which is very different from the design strategy that Intel or IBM have used in their own solutions. And, always according to Cerebras, to optimize the execution of deep learning algorithms it is necessary to bet on a chip equipped with a very high intrinsic parallelism that is manifested through the packaging of a huge number of cores. This is why the WSE chip incorporates the astonishing number of 400,000 programmable cores.
However, not the entire surface of the chip is dedicated to the process cores, of course. Another of the subsystems that also monopolizes an important part of logic is memory. Placing it close to the cores reduces latency, increases performance and minimizes consumption in a perceptible way, again according to Cerebras. And it makes sense. The WSE chip integrates 18 GB memory.
And, to conclude, two more figures that strengthen the ambition of this processor: the memory bandwidth is close to 9.6 Petabytes, and the 400,000 cores communicate with the outside through a link with a speed of transfer of 100 Petabits per second. They are monstrous figures that are very far from those handled by the processors of our computers. Of course, we must not lose sight of the fact that WSE chips are not good for anything. Theirs is deep learning.
Images | Cerebra
More information | Cerebra