TIER III liquid-cooled data center is it real!

Introduction

The primary goal of a data center is to provide end users with reliable, high-performance access to applications, data, and services, minimizing downtime and ensuring robust data security.

Data Center Functions

Availability in a data center refers to the constant accessibility and responsiveness of services, applications, and data for end users. This involves ensuring optimal application performance and uninterrupted data access with minimal downtime. High availability is achieved through redundant systems, failover mechanisms, and proactive monitoring to detect and resolve issues before they affect users. This approach helps maintain business continuity, ensuring that users can rely on the data center to support critical operations without interruptions, thus maximizing productivity and satisfaction.

Architecture's Obligations

The compute architecture of an AI data center is designed to meet the immense processing and storage demands of AI workloads. It includes high-performance CPUs, GPUs, specialized AI accelerators, and a robust interconnect network for efficient data transfer between components. The architecture is optimized for parallel processing, enabling rapid execution of complex algorithms and models, and incorporates scalable storage solutions to handle vast amounts of data needed for training and inference. Additionally, it emphasizes redundancy, fault tolerance, and energy efficiency to ensure continuous operation and sustainability, supporting cutting-edge AI research, development, and deployment.

Shifting Stakeholders and Standards Bodies mindset

Data center facilities are driven by IT infrastructure requirements and the application layer requirements, where end users interact with services. Thus, the concept of availability should align with the application layer; if the application layer is available, the data center is considered available. Although cooling and power are not the primary services provided to end users, they remain crucial and must be considered during the planning and design phase. Cluster computing for cloud and AI applications is a key topology in hyperscale data center architecture, with applications deployed over groups of servers and IT hardware.

Liquid Cooling Necessity

Liquid cooling has become essential for high-performance computing (HPC) hardware providers, particularly in AI applications, due to its superior efficiency in managing the heat generated by advanced computational processes. As AI models become more complex and power-hungry, traditional air cooling methods often fall short, leading to thermal throttling and reduced performance. Liquid cooling systems dissipate heat more effectively, ensuring hardware operates at optimal speeds without overheating, enhancing reliability and longevity. This supports the high computational demands of AI workloads, making liquid cooling indispensable in modern HPC infrastructure.

Concurrent Maintainability in Liquid Cooling Design

Concurrent maintainability involves conducting planned maintenance without service disruption by providing redundant components and paths. As applications are the core service provided by data centers, stakeholders should focus on application availability over server clusters, not limited to cooling and power redundancy. Deploying applications over a multi-cluster environment allows for concurrent maintainability in liquid-cooled data centers, as each server can be provisioned in different tanks for full immersion cooling or in separate racks for direct-to-chip scenarios. Integrating IT architecture and facility planning ensures that a liquid-cooled data center meets Tier III certification requirements.

Tier III Liquid Cooling Design Considerations

As of now, the Uptime Institute has not certified a liquid-cooled data center but has announced the minimum requirements designers should consider.

In liquid cooling systems, the Coolant Distribution Unit (CDU) is crucial for managing coolant flow and distribution. For immersion cooling, the CDU removes heat from submerged servers, while in direct-to-chip cooling, it regulates coolant flow to cold plates attached to processors and other heat-generating components. Ensuring redundant CDUs enhances cooling efficiency, reliability, and data center performance. Failure of a CDU can paralyze the liquid cooling system, so redundancy in components like circulating pumps is essential.

Cooling Fluids

Two common types of cooling fluids in liquid cooling systems are perfluorocarbon (PFC) based liquids and synthetic oil-based liquids. PFC-based liquids are non-conductive and have excellent heat transfer properties but are more expensive and have a higher global warming potential. Synthetic oil-based liquids are less costly and have a lower environmental impact but can be combustible at high temperatures. A single point of failure in concurrent maintainability scenarios can occur if there is a lack of redundancy in cooling system components, leading to potential overheating and system downtime.

Conclusion

Building a concurrently maintainable Tier III liquid-cooled data center facility is achievable by integrating the application layer with IT high-availability cluster infrastructure, supported by redundant CDUs, liquid-cooled tanks, or direct-to-chip racks.