Deep Learning Programming at Scale

Deep learning has become one of the most important computational workloads of our generation, advancing applications across industries from healthcare to autonomous driving. But it is also profoundly computationally intensive. (Updated June 2022.)

Powering Extreme-Scale HPC with Cerebras WaferScale Accelerators

In this paper, we will explore the challenges facing HPC developers today and show how the Cerebras architecture can help to accelerate sparse linear algebra and tensor workloads, stencilbased partial differential equation (PDE) solvers, N-body problems, and spectral algorithms such as FFT that are often used for signal processing.

The Cerebras Software Development Kit: A Technical Overview

Cerebras has introduced a new software development kit (SDK) which allows anyone to take advantage of the strengths of the CS-2 system. Developers can use the Cerebras SDK to create custom kernels for their standalone applications or modify the kernel libraries provided for their unique use cases. The SDK enables developers to harness the power of wafer-scale computing with the tools and software used by the Cerebras development team.

Training Giant Neural Networks Using Weight Streaming on Cerebras Wafer-Scale Systems

In this paper, we survey existing approaches used to scale training to clusters of compute units and explore the limitations of each in the face of giant models. We present a new paradigm for giant model training, called weight streaming, whiche enables the training of models two orders of magnitude larger than the current state-of-the-art, with a simple scaling model.

Cerebras Systems Enables Brain-scale AI

This research paper explores Cerebras System's approach to create a brain-scale AI and the new technologies that could enable that feat. But first, let's put this discussion into the proper context. Just how big is a 120 trillion-parameter model?

Limits to Scale-Out for Training Language Models

Natural language processing has revolutionized how data is consumed, meaning that computational demand has skyrocketed. Companies in every industry are using graphics processing unit (GPU) clusters to keep up. But is this really the best solution?

Train Large BERT Models Faster with Cerebras Systems

Unstructured text is one of the largest human-generated data sources. Web data, academic publications, emails, traditional media, texts, instant messages, digital records, social media — all hold an enormous volume of unstructured text.

Cerebras Systems: Achieving Industry Best AI Performance Through A Systems Approach

The CS-2 is a system solution that consists of innovations across three dimensions: a) the second
generation Cerebras Wafer Scale Engine (WSE-2) — the industry’s largest and only multi-trilliontransistor processor, b) the Cerebras System and c) the Cerebras software platform.