Elon Musk's AI venture,
xAI
, has commenced training its large language model (LLM),
Grok
, on what he claims is the
world's most powerful AI
training cluster. Located in Memphis, Tennessee, the system is equipped with 1,00,000 Nvidia
H100 AI chips
. Musk made the announcement on X (formerly Twitter), his other company.
“Nice work by @xAI team, @X team, @Nvidia & supporting companies getting Memphis Supercluster training started at ~4:20am local time,” he said in a post.
“With 100k liquid-cooled H100s on a single RDMA fabric, it’s the most powerful AI training cluster in the world!” Musk added.
In a thread, Musk noted that the cluster will give his company’s AI model “a significant advantage”.
“This is a significant advantage in training the world’s most powerful AI by every metric by December this year,” he said.
Earlier in the day, he said that “Grok is being trained in Memphis”.
Why this is important
The development comes about two weeks after a report said that xAI and Oracle ended talks on a potential $10 billion server deal. The report added that Musk is building its own data centre and is buying AI chips for it – something Musk later confirmed via a post on X.
“xAl is building the 100k (100,000) H100 system itself for [the] fastest time to completion. Aiming to begin training later this month. It will be the most powerful training cluster in the world by a large margin,” he added.
"The reason we decided to do the 100k H100 and next major system internally was that our fundamental competitiveness depends on being faster than any other AI company. This is the only way to catch up," Musk highlighted.
According to Musk, xAI will release Grok 2 in August, and Grok 3 will be available in December.
"Grok 2 is going through finetuning and bug fixes. Probably ready to release next month," Musk added.
