logo
logo
  • Product
    • System
    • Chip
    • Software
    • Cloud
  • Industries
    • Health & Pharma
    • Energy
    • Government
    • Scientific Computing
    • Financial Services
    • Web and Social Media
  • Resources
    • Customer Spotlights
    • Blog
    • Publications
    • Events
    • White Papers
  • Developers
    • Community
    • Developer Blog
    • Documentation
    • ML Public Repository
    • Request Access to SDK
  • Company
    • About
    • In the News
    • Awards
    • Press Releases
    • Press Kit
  • Join Us
    • Life at Cerebras
    • All openings
  • Get Demo
  • Search
October 31, 2019
In Chip, Machine Learning, Software, Blog

Cerebras speaks at the TensorFlow World 2019 Conference

On October 30th at TensorFlow World, my colleague Manjunath Kudlur and I were honored to speak publicly for the first […]

On October 30th at TensorFlow World, my colleague Manjunath Kudlur and I were honored to speak publicly for the first time about the Cerebras software stack. This stack links deep learning researchers to the massive compute capabilities of the Wafer Scale Engine (WSE).

The WSE is the largest commercial chip ever manufactured, built to solve the problem of deep learning compute. The WSE is 1.2 trillion transistors, packed onto a single chip with 400,000 AI-optimized cores, connected by a 100Pbit/s interconnect. The cores are fed by 18 GB of super-fast, on-chip memory, with an unprecedented 9 PB/s of memory bandwidth.

What does this mean for AI researchers? We believe, along with many others, that AI has massive potential — from ads, to autonomous vehicles, from commerce to climate. It has transformative potential for the way we live and work.

Researchers continue to see gains with deeper models and larger datasets. But, they are compute limited today. Training commonly takes days, weeks, even months. Not only is this costly, it constrains research and development. We need wall clock training times on the timescale of experimentation and human innovation — minutes-hours rather than days-weeks — even for large models. This means we need 100-1000x increase in compute capabilities, not incremental 1.5-2x.

We need this performance in an accessible, easy to program package.

This talk introduced the Cerebras Software Stack. Its primary task is to map the computational graph for researchers’ neural networks — from the framework level all the way down to our massive wafer-scale processor.

The stack integrates seamlessly with popular machine learning frameworks like TensorFlow and PyTorch, allowing researchers to use familiar and flexible tools that bring their models to the WSE.

A programmable C++ interface allows researchers to extend the platform and develop custom kernels – empowering them to push the limits of ML innovation.

To learn more, check out the slides here, or contact us – we’re a friendly bunch! Review our careers page if you’re interested in joining our incredible team.


Ugnius
Author posts
Related Posts
ChipMachine LearningSoftwareBlog

August 15, 2022

Context is Everything: Why Maximum Sequence Length Matters

GPU-Impossible™ sequence lengths on Cerebras systems may enable breakthroughs…


by Sia Rezaei

ChipMachine LearningBlog

August 3, 2022

Cerebras Wafer-Scale Engine Inducted into the Computer History Museum

Today was a proud day for the entire Cerebras family.


by Rebecca Lewington

Machine LearningSoftwareBlog

June 22, 2022

Cerebras Sets Record for Largest AI Models Ever Trained on Single Device

Our customers can easily train and reconfigure GPT-3 and GPT-J language models…


by Joel Hestness

Add comment Cancel reply

You must be logged in to post a comment.

  • Prev
  • Next

Explore more ideas in less time. Reduce the cost of curiosity.

Sign up

info@cerebras.net

1237 E. Arques Ave
Sunnyvale, CA 94085

Follow

Product

System
Chip
Software
Cloud

Industries

Health & Pharma
Energy
Government
Scientific Computing
Financial Services
Web & Social Media

Resources

Customer Spotlight
Blog
Publications
Event Replays
Whitepapers

Developers

Community
Developer Blog
Documentation
ML Public Repository
Request Access to SDK

Company

About Cerebras
In the News
Press Releases
Privacy
Legal
Careers
Contact

© 2022 Cerebras. All rights reserved

Privacy Preference Center

Privacy Preferences

Manage Cookie Consent
To provide the best experiences, we use technologies like cookies to store and/or access device information. Consenting to these technologies will allow us to process data such as browsing behavior or unique IDs on this site. Not consenting or withdrawing consent, may adversely affect certain features and functions.
Functional Always active
The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.
Preferences
The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.
Statistics
The technical storage or access that is used exclusively for statistical purposes. The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.
Marketing
The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.
Manage options Manage services Manage vendors Read more about these purposes
View preferences
{title} {title} {title}