Workload-Dependent Thermal Locality in AI GPU Nodes: Evidence from H100 and B200 Intra-Node Thermal Divergence

Abstract

The increasing power density of AI infrastructure places critical importance on the thermal behavior of data-center GPUs. Measurements of these GPUs’ temperatures, however, rely on aggregate metrics such as hardware power ratings, data-center efficiency metrics, and node temperatures, none of which provide information about the temperature distribution across the GPUs within that node. As a result, it is important to determine whether thermal divergence among GPU nodes varies with the hardware model alone or with the workloads assigned to those nodes. An analysis of 32 public sessions, each with eight GPUs assigned to either image-generation or language-model and text-generation workloads, shows that the mean temperature spread between the hottest and coolest GPU within each session is 13.58˚C, a value that far exceeds the 1˚C resolution of each GPU’s temperature measurements. Furthermore, analysis of the interaction between workload and hardware reveals that H100 GPUs exhibit a greater temperature spread during image-generation workloads than during language-model and text-generation workloads. In comparison, B200 GPUs exhibit a greater temperature spread during language-model and text-generation workloads than during image-generation workloads (interaction effect: 16.77˚C, F(1, 28) = 67.88, p < 0.001). These results are statistically significant even after adjusting for heteroskedasticity, permutation tests, bootstrap resampling, leave-one-session-out model refitting, and total GPU power within each session. Furthermore, because each set of GPU workload data was collected on different dates, these results indicate only a relationship between workload and GPU thermal divergence, not between the workload itself and that divergence. Thus, thermal assumptions based solely on the hardware model or on the average temperature of GPUs within a node provide incomplete descriptions of the thermal behavior of those nodes. As such, decisions regarding data-center cooling, scheduling workloads across nodes, hardware procurement, and managing the thermal behavior of those GPUs should account for thermal divergence based on each GPU’s workload. Furthermore, while this study was unable to provide evidence of any effect of these workloads on the throttling or useful computation of those GPUs, such conclusions would require validation of these assumptions.