AI Supercomputer

NCCL AllReduce traffic across N 8-GPU nodes is most efficient when it is aggregated within the same rail (GPUs at the same index). In a standard fat-tree, ECMP hash collisions scatter same-rail traffic across different spines, dropping effective bandwidth to 60–70% of spec. A Rail-Optimized topology + adaptive routing keeps same-rail traffic on the same path, recovering 95%+ efficiency. The benefit is greatest at ≥ 64 8-GPU nodes (≥ 512 GPUs).

LLM training produces burst I/O every epoch or step, as N GPUs write checkpoints simultaneously. Standard NFS/SMB storage cannot absorb these bursts, leaving GPUs idle waiting on I/O. The combination of NVMe-oF (RDMA-based) + a Spectrum Scale backend hides inter-step I/O to 0% of GPU time, raising utilization to 78–85% in 70B-model training. We also evaluate checkpoint compression and shard distribution.

The NVIDIA Blackwell B300 delivers ~20 PFLOPs FP8 per GPU with 288 GB of HBM3e — more than enough for both LLM training and inference. Eight GPUs per node secures 1.8 TB/s of bidirectional inter-GPU bandwidth via NVLink/NVSwitch, while striking the balance point for single-node PCIe, memory channels, and power (within liquid-cooling limits). Some sites also run H100 × 8 / H200 × 8 alongside, separating pools by workload.