IBM Cloud is broadening its AI technology services with Intel Gaudi 3 AI accelerators now available to enterprise customers.
With Gaudi 3 accelerators, customers can more cost-effectively test, deploy and scale enterprise AI models and applications, according to IBM, which is said to be the first cloud service provider to adopt Gaudi 3.
Intel Gaudi 3 accelerators include AI-specific designs and features and are targeted at multi-model large language models (LLMs) and retrieval-augmented generation (RAG) development, IBM stated. The accelerators feature matrix math engines, tensor processing cores, high-bandwidth memory, and built-in Ethernet ports for accelerated inferencing of deep neural networks, according to IBM.
“Enterprises can scale from a single node (eight accelerators) with a throughput of 9.6 TB/s to a 1,024-node cluster (8,192 accelerators) with a throughput of 9.830 PB/s. Scaling is achieved using a choice of numerous industry-standard and high-capacity Ethernet switches and other supporting infrastructure to help lower costs,” IBM stated.
Intel Gaudi 3 can be deployed through IBM Cloud Virtual Servers for virtual private cloud (VPC) instances. Businesses can pick their compute, storage and networking resources as needed, IBM stated.
In addition, the package can be provisioned as a container worker node, and IBM Cloud said it has plans to deliver Intel Gaudi 3 as a worker node for Red Hat OpenShift AI clusters and Red Hat OpenShift on IBM Cloud in early 2025. The Intel Gaudi 3 instance also supports Red Hat Enterprise Linux AI image options.
For businesses that need more control over their AI development, IBM says they can deploy IBM watsonx.ai software with the Intel Gaudi 3-based virtual server on IBM Cloud VPC in Q2 2025. IBM watsonx.ai includes an end-to-end AI development studio, AI developer toolkit and full AI lifecycle management for developing AI services and deploying them into customers applications.
“As AI is increasingly moving from an experimental trend to the backbone of real world applications, IT organizations are challenged with balancing the necessary performance with economic considerations of AI hardware, and doing so at scale,” wrote Mitch Lewis, a performance analyst with Signal65, which offers tech industry testing, performance validation, and data-based consulting.
“Previous analysis by Signal65 demonstrated that Intel Gaudi 3 accelerators were capable of offering highly competitive performance for AI inferencing workloads, while offering substantial economic advantages. The availability of Gaudi 3 accelerators on IBM Cloud looks to build upon these advantages while providing IT organizations with an easily accessible and scalable cloud-based approach to deploying AI applications,” Lewis wrote in a blog post about the Intel Gaudi 3 AI accelerator implementation on IBM Cloud.
“This preliminary performance testing conducted by Signal65 found Intel Gaudi 3 to offer highly competitive performance when compared to alternative Nvidia-based offerings on IBM Cloud. Gaudi 3 on IBM Cloud provides a flexible platform capable of achieving high performance across various models and technical configurations,” Lewis wrote. “In addition, the pricing of Gaudi 3 instances on IBM Cloud builds an appealing economic advantage over both Nvidia instance types” – the Nvidia H100 and H200 – that IBM Cloud also supports, Lewis stated.