01Challenge
The situation
The prior PoC delivered only 60% NCCL all-reduce efficiency. Checkpoint I/O consumed 20% of training time. Multi-tenancy was unresolved.
CASEEnterprise AI Division
TIER T2300-node H100 cluster deployment
Workload
LLM Training (~70B)
Scale
300 H100 nodes / NDR InfiniBand / 12PB storage
02Approach
The approach
- 01Rail-Optimized topology + adaptive routing
- 02NVMe-oF + Spectrum Scale with separated metadata/data NSDs
- 03Slurm fair-share + priority queues
- 04Luxe Orbit management server redundancy + automated OS deployment
03Results
Results
NCCL all-reduce
94-97% peak
Checkpoint I/O share
< 2%
Deployment timeline
14 weeks
04Suite Used
Operations automation — Luxe Series products used
- Luxe Orbit
Have a similar case?
Just tell us your workload type and scale, and we'll give you a first diagnosis within 30 minutes.
Request a consultation →