The Total Cost of AI Infrastructure

A purchase order captures a fraction of what AI infrastructure costs. Modelled across five years, capital equipment is frequently the minority of the total.

The five components

Capital covers accelerators, networking, storage, racks and cooling equipment. Power covers both the draw of the equipment and the facility overhead of removing its heat, which in this climate is a substantial multiplier rather than a rounding error. Floor space is either rent or opportunity cost. Staff covers the people who operate the cluster, which is routinely omitted entirely. Facility work covers power distribution, cooling capacity and floor loading, which is usually a one-off but is often unbudgeted because it was never scoped.

What utilisation does to the model

Utilisation dominates cost per useful hour more than any other variable. Idle accelerators still draw a significant fraction of peak power, and facility overhead is largely fixed, so a cluster at forty percent utilisation pays close to the same running cost as one at eighty while producing half the work. Cost per useful hour roughly doubles.

This is why moving development work off the production cluster has a larger financial effect than most procurement negotiations. It raises the denominator.

Where the model usually goes wrong

Two errors recur. Sizing the fabric and storage from whatever budget remains after the accelerators, which produces a cluster that cannot be kept busy and therefore fails on the utilisation term above. And treating facility work as a facilities problem rather than a project cost, which hides it until it appears on the critical path.

The more reliable approach is to budget from the rack inward: establish what the facility will commit to, size the data path to keep the intended node count fed, and let the accelerator count be what the remainder supports.