We're Gonna Need a Bigger GPU
Kif Leswing
NVIDIA is clearly wasting no time resting on their lofty laurels:1
The new generation of AI graphics processors is named Blackwell. The first Blackwell chip is called the GB200 and will ship later this year. Nvidia is enticing its customers with more powerful chips to spur new orders. Companies and software makers, for example, are still scrambling to get their hands on the current generation of “Hopper” H100s and similar chips.
“Hopper is fantastic, but we need bigger GPUs,” Nvidia CEO Jensen Huang said on Monday at the company’s developer conference in California.
As we ride into the sunset of Moore's Law, we've been conditioned to expect CPU upgrades to be small speed bumps, but no longer the huge performance gains of yesteryear. GPUs, it seems, are a different story:
Every two years Nvidia updates its GPU architecture, unlocking a big jump in performance. Many of the AI models released over the past year were trained on the company’s Hopper architecture — used by chips such as the H100 — which was announced in 2022.
Nvidia says Blackwell-based processors, like the GB200, offer a huge performance upgrade for AI companies, with 20 petaflops in AI performance versus 4 petaflops for the H100. The additional processing power will enable AI companies to train bigger and more intricate models, Nvidia said.
Yes, that's a 5x change in petaflops. But what about practical usage in real-world AI? Well, when put together in a new rack:
Nvidia said that the system can deploy a 27-trillion-parameter model. That’s much larger than even the biggest models, such as GPT-4, which reportedly has 1.7 trillion parameters. Many artificial intelligence researchers believe bigger models with more parameters and data could unlock new capabilities.
1 NVIDIA is getting dangerously close to passing Apple -- yes, Apple -- in market cap. Forget Intel, currently, Apple sits at $2.68T while NVIDIA is at $2.21T. In fact, they're basically as close to Apple as Apple is to current #1 Microsoft ($3.10T). Apple had a nice boost early on from their own AI-related report today, before settling back to reality. How much would Jensen Huang like to pass Apple, do we think?