Wafer-Scale Engine (WSE)

The Cerebras Wafer Scale Engine (WSE) is a single chip that is optimized for Artificial Intelligence work. The WSE is the largest chip ever built. It is the industry’s only trillion transistor processor, and contains more cores, more local memory, and more fabric bandwidth than any chip in history. This enables fast, flexible computation at lower latency and with less energy. 

The WSE covers 46,255 square millimeters – 56 times larger than the largest graphics processing unit. With 400,000 cores, 18 Gigabytes of on-chip SRAM, 9.6 petabytes/sec of memory bandwidth, and 100 petabits/sec of interconnect bandwidth, the WSE contains 78 times more compute cores, 3,000 times more high-speed on-chip memory, 10,000 times more memory bandwidth and 33,000 times more fabric bandwidth than its graphics processing competitor. In effect, it provides the compute capacity of an entire cluster in a single chip, without the cost, complexity, and bandwidth bottlenecks involved with lashing together hundreds of smaller devices. This enables fast, flexible computation, at lower latency and with less energy — unlocking unprecedented deep learning performance on a single chip. 

In AI, chip size is profoundly important. Big chips process information more quickly, producing answers in less time. Reducing the time-to-insight, or “training time,” allows researchers to test more ideas, use more data, and solve new problems. Google, Facebook, OpenAI, Tencent, Baidu, and many others argue that the fundamental limitation to today’s AI is that it takes too long to train models. Reducing training time removes a major bottleneck to industry-wide progress. 

“Designed from the ground up for AI work, the Cerebras WSE contains fundamental innovations that advance the state-of-the-art by solving decades-old technical challenges that limited chip size—such as cross-reticle connectivity, yield, power delivery, and packaging,” said Andrew Feldman, founder and CEO of Cerebras Systems. “Every architectural decision was made to optimize performance for AI work. The result is that the Cerebras WSE delivers, depending on workload, hundreds or thousands of times the performance of existing solutions at a tiny fraction of the power draw and space.”