AMD (NASDAQ: AMD) has announced that Oracle Cloud Infrastructure (OCI) will be using AMD Instinct™ MI300X accelerators with ROCm™ open software for its newest Compute Supercluster instance, named BM.GPU.MI300X.8. This powerful setup is designed to handle massive AI models, with the ability to connect up to 16,384 GPUs in a single cluster, utilizing the same high-speed network technology as other OCI accelerators.
These bare-metal instances are optimized for AI applications such as large language model (LLM) training and inference, which demand high memory capacity and bandwidth. Fireworks AI is among the early adopters of this technology.
“AMD Instinct MI300X and ROCm open software are proving themselves as key components for handling critical AI workloads on OCI,” said Andrew Dieckmann, Corporate Vice President and General Manager of it’s Data Center GPU Business. He emphasized that this collaboration is driving better performance, efficiency, and flexibility for OCI’s AI-intensive customers.
Donald Lu, Senior Vice President of Software Development at OCI, added, “The AMD Instinct MI300X accelerators bring powerful inference capabilities, enhancing our broad range of high-performance bare metal instances, and offering customers a competitive edge in AI processing.”
Performance and Flexibility for AI Training and Inference
The AMD Instinct MI300X underwent rigorous testing by OCI, proving its ability to handle large-scale AI models, particularly in latency-sensitive applications and larger batch sizes. The ability to manage the largest LLM models in a single node has attracted attention from AI developers.
Fireworks AI, a platform focused on building and deploying generative AI models, is already benefiting from the performance of the AMD Instinct MI300X on OCI. “With over 100 models, we’re able to scale our services to meet growing demands,” said Lin Qiao, CEO of Fireworks AI, praising the accelerators’ memory capacity and scalability.
For more than five decades, AMD has been a leader in high-performance computing, graphics, and visualization technologies. Its technology is relied upon by billions of people, top Fortune 500 companies, and research institutions worldwide, driving innovation in various industries. It’s mission is to develop cutting-edge products that redefine the limits of computing performance.
