What are the impacts of artificial intelligence on data centers?

AI: a rich history going back further than you might think

In 1950, Alan Turing posed a fundamental question: “Can machines think?” He proposed the “Turing test,” an experiment designed to determine whether a machine can demonstrate human intelligence and be indistinguishable from a human in the responses given to a question. This experiment laid the groundwork for inquiry into AI. The next few decades, however, saw slow progress, largely due to insufficient technical means.

In the 1990s, AI enjoyed a new boom thanks to progress in computing and the birth of the Internet and Big Data, which enabled broad collaborations and the production of the volumes of data necessary to effectively train algorithms. Since then, AI has become a key tool of the 4.0 era, with applied research in fields as diverse as research, health, finance, office automation, industrial modeling, graphic design, translation, computer programming, etc.

Machine learning and deep learning are aimed at creating systems capable of solving problems autonomously, based on the principles of rationality and imitation of human intelligence.

The AI market entered a phase of very active growth with the release of Chat GPT 3.5 in 2022. This is a type of AI called generative AI, which applies base models to create original content (text, images, structures, etc.). Since its launch, several increasingly high-performing generative AI tools have been born, supported by significant financial resources.

The essential evolution of data centers in the age of AI

To process large quantities of data quickly in order to solve complex problems like generating text, images, or 2D or 3D plans, AI requires servers with specific powerful processors. This is why, in data centers, the traditional CPU (Central Processing Unit) is being replaced by the GPU (Graphic Processing Unit), which performs operations and calculations 100 times faster than the CPU for AI applications, or the TPU (Tensor Processing Unit), an integrated circuit developed by Google specifically to accelerate AI systems. The TPU performs around 2.5 times better than the GPUs currently in use in the supercomputers (HPC systems) of the GAMAM companies. It is estimated that thousands or even tens of thousands of GPUs are required to train the largest AI models like GPT-3 and GPT-4.

The race for power is only just beginning: the new Blackwell chip unveiled on March 18, 2024, by NVIDIA is intended to support AI growth with unprecedented capacities.

These new chips integrated into servers to run AI technology are radically changing ways of thinking about, designing, building, and, most importantly, cooling data centers and supplying them with electricity.

Therefore, today’s data centers must be designed and operated differently in order to meet the requirements of AI. Players in the data center industry anticipated this and are adapting to these new requirements.

1. New cooling systems

Rack density, or the power consumed by the equipment in a server rack, is where one of the main differences lies between traditional data centers and those dedicated to AI. Multipurpose data centers have average rack power densities of 5 to 15 kilowatts (kW), while in AI data centers, this is many times higher and can exceed 100 kW per rack. These kinds of power densities mean that AI cannot be run in a “classic” data center, because the air exchange capacity isn’t sufficient to dissipate the heat energy released.

New, more efficient cooling systems are necessary, such as direct liquid cooling (DLC), generally using water, and immersion cooling, in which servers are immersed in oil baths to dissipate the heat produced. It should be noted that the water circulates around a closed loop, which results in very low water consumption, limited to the water needed to fill the cooling circuit. A good number of existing and planned French data centers are currently being adapted to these cooling techniques. Other avenues, such as DLC using gas or liquid metal, are also being explored.

2. Hybrid infrastructure

The infrastructure used to train AI models doesn’t have the same requirements in terms of service continuity as the infrastructure used in inference data processing centers. The trend that seems to be emerging is hybridization: carrying out training in a smaller, denser data center and inference in a less dense, even classic data center.

In the age of AI, data centers require more flexible, modular, and adaptive infrastructure. In short, hybrid infrastructure.

This could also mean bringing AI processing as close as possible to users, for example with SLMs (Small Language Models) on smartphones. A network of edge data centers spread out geographically, and therefore closer to the users’ machines, can lighten the loads of hyperscale data centers and reduce latency.

3. The challenge of exponentially increasing energy consumption

The data center industry is making a particular effort to reduce energy consumption. The average data center PUE (Power Usage Effectiveness, standard energy efficiency metric: the closer it is to 1, the more efficient the data center is) went from 2.5 in 2005 to 1.58 in 2023 [Source for average PUE]. Data center acceptability depends on the availability of electricity supplies being maintained for all users. A close partnership with the French operator RTE is therefore necessary to balance and anticipate needs. According to a recent study by Ademe, data centers only accounted for 16% of the carbon footprint of digital technology in France in 2022 (compared to 79% for terminals).

According to a study by the Uptime Institute, the majority of racks currently used for (generative) AI training and inference are racks that consume less than 20 kW. This means that generative AI (accounting for 20% of the use of servers and infrastructure worldwide) is not currently synonymous with extreme densities and therefore huge electricity demand.

It is, however, important to anticipate a potential significant increase in demand in the future. The IAE have predicted that energy demand from artificial intelligence and cryptocurrency could double between 2026 and 2030, and the increase could even be greater than that according to certain scenarios. This inevitably raises the question of the additional energy production and its impact on the environment and society as a whole.

Many ideas for ways of controlling and reducing these problematic predicted consumption levels have already been raised. To list just a few of these:

• Eliminating consumption at peak times to give the grid flexibility (data centers are usually equipped with independent energy production solutions such as biofuel or gas generators).
• Trigeneration, technology used to generate cooling, heat, and power from natural gas.
• Free cooling, which drastically reduces the energy needed to cool the servers.
• Recovery of waste heat for users who need it, which in effect reduces their energy consumption.
• Moving data processing from one data center to a different one in a different geographical area to reduce local electricity consumption (flexibility enabled by the cloud).
• Potential, and careful, use of small modular reactors (SMRs) with an output of up to 300 MW electric.
• Bringing data processing closer to users with SLMs.

A need to rationalize AI use and AI data centers

Although AI necessitates rapid growth in energy consumption, it is already bringing about important progress in efficiency and productivity. In the key sector of clinical research, AI enabled the COVID-19 vaccine to be developed extremely fast.

Without the computing and processing power of data centers, researchers wouldn’t have been able to process and analyze billions of pieces of data in record time. It’s therefore thanks to AI (and data centers) that the vaccines were able to be developed and put on the market as quickly as they were during the COVID-19 pandemic.

AI has enormous potential, and the challenges associated with its use are only just beginning. They affect all stakeholders, both data center players and end users. The former must continue their efforts to reduce energy consumption and environmental impact by means of the innovation that has always characterized the sector, while the latter are stakeholders and are responsible for the increase in consumption. It is therefore essential that all parties endeavor to be economical and responsible in their use of the high-performing services AI offers on a day-to-day basis.