Accelerating Large Language Model Training with Variable Sparse Pre-training and Dense Fine-tuning
We reduced pre-training FLOPs by 64% using sparsity. To the…
Variable Sequence Length Training for Long-Context Large Language Models
We show it is possible to accelerate the training for large…
SlimPajama: A 627B token, cleaned and deduplicated version of RedPajama
Today we are releasing SlimPajama – the largest…
Efficient Large-Scale GPT Training Using a Cerebras Wafer-Scale Cluster
Cerebras has built a platform for push-button training of…
Cerebras Architecture Deep Dive: First Look Inside the HW/SW Co-Design for Deep Learning [Updated]
Our ML-optimized architecture enables the largest models to…
Fine-Tuning with Cerebras AI Model Studio Launchpad
Cerebras shares research showing smaller foundation models…
Cerebras-GPT: A Family of Open, Compute-efficient, Large Language Models
Cerebras open sources seven GPT-3 models from 111 million…
Cerebras Announces Fine-Tuning on the Cerebras AI Model Studio
Announcing the addition of fine-tuning capabilities for…
Real-Time Computational Physics with Wafer-Scale Processing [updated]
Cerebras and NETL achieve two orders of magnitude…
To Bfloat or not to Bfloat? That is the Question!
The bfloat16 data format for deep learning shortens…
Harnessing the Power of Sparsity for Large GPT AI Models
Enabling innovation of novel sparse ML techniques to…
Genomics in Unparalleled Resolution: Cerebras Wafer-Scale Cluster Trains Large Language Models on the Full COVID Genome Sequence
Our joint work with Argonne National Laboratory (ANL) and…
Context is Everything: Why Maximum Sequence Length Matters
GPU-Impossible™ sequence lengths on Cerebras systems may…
Training Multi-Billion-Parameter Models on a Single Cerebras System is Easy
Changing model size is trivial on Cerebras, rather than a…
Multi-Billion-Parameter Model Training Made Easy with CSoft R1.3
CSoft R1.3 delivers GPT-J continuous pre-training, more…
