xAI Launches World’s Largest AI Training System

The AI startup xAI has unveiled its Colossus AI training cluster, utilizing 100,000 Nvidia H100 graphics processors, as announced by the company’s head, Elon Musk.

This weekend, the @xAI team brought our Colossus 100k H100 training cluster online. From start to finish, it was done in 122 days.

Colossus is the most powerful AI training system in the world. Moreover, it will double in size to 200k (50k H200s) in a few months.

Excellent…

— Elon Musk (@elonmusk) September 2, 2024

“Colossus is the most powerful AI training system in the world. Moreover, it will double in size to 200,000 (50k H200). Excellent work by the team, Nvidia, and our many partners/suppliers,” added the entrepreneur.

According to Musk, the cluster was deployed in 122 days.

In July, Musk revealed plans to launch a system with 100,000 Nvidia H100 processors for AI training.

At the end of August, xAI faced allegations of environmental pollution in Memphis, Tennessee, due to the use of gas turbines for a new data center.

Here is a comparison chart to help everyone understand the magnitude of this. pic.twitter.com/PJys0XlvYo

— Anthony Everywhere (@AnthonyEveryWhr) September 2, 2024

OpenAI’s flagship model uses 80,000 graphics processors, while Google’s employs 90,000.

Nvidia highlighted Colossus’s “breakthrough performance and exceptional energy efficiency gains.”

Exciting to see Colossus, the world’s largest GPU #supercomputer, come online in record time. Colossus is powered by @nvidia‘s #acceleratedcomputing platform, delivering breakthrough performance with exceptional gains in #energyefficiency.

Congratulations to the entire team! https://t.co/UXHtPCELly

— NVIDIA Data Center (@NVIDIADC) September 2, 2024

Previously, Musk showcased a new “giant supercluster” for AI training and operations, under construction at Tesla’s headquarters in Austin.

In August, xAI introduced Grok-2 with an uncensored image generator.