PCIe or SXM: What the Choice Actually Costs

Two eight-GPU servers can differ by more than price. One holds its accelerators in PCIe slots; the other has them soldered to a baseboard with a dedicated interconnect between them. That is not a tier, it is a different machine for a different job.

What SXM buys

A baseboard design puts a high bandwidth link directly between every GPU. When a model is too large for one card and has to be split across eight, every step of training moves activations between them, and that traffic is the limit rather than the arithmetic. On PCIe the same traffic crosses a bus shared with everything else.

So for training a model that spans the whole node, the interconnect is the specification that decides throughput. Nothing else on the datasheet compensates for it.

What PCIe buys

A card in a slot can be removed. It can be replaced with a different card, replaced after a failure without returning the chassis, or added to later. A slot not holding a GPU holds a network adapter or a storage controller instead.

That flexibility is worth more than interconnect bandwidth for inference, where models usually fit on one or two cards and the accelerators are working independently rather than in lockstep. It is also worth more when the deployment has to survive five years of changing requirements, because the machine can be reconfigured rather than replaced.

The question that settles it

Does one job need all the GPUs at once, or do many jobs each need one or two?

Training a large model is the first. Serving many models, or many copies of one model, is the second. Most estates do both, which is why a mixed fleet is common and why the two are priced against each other rather than one being the upgrade.

What we ask before quoting

The largest model you intend to train, and whether it fits in the memory of a single accelerator. If it does, PCIe is very likely the better value. If it does not, the interconnect stops being a line item and becomes the design.