Stream Data Centers Articles

Managing AI Deployments in the Real World

Managing AI Deployments in the Real World

AI has gone mainstream, and the impact is reverberating everywhere. We’ve been watching the trend develop and have worked hard to prepare ourselves to respond—including through proactive land development, standardized (and configurable) data center design, and flexible deal structures.

Widespread adoption of artificial intelligence (AI) has been ‘just around the corner’ for decades. (Researchers have been working on neural networks since the early 1970s.) Deep Blue beat Garry Kasparov in chess in 1997 but it wasn’t until 14 years later that Watson won Jeopardy. Another decade later, and AI is driving cars, writing code, and making medical diagnoses. (And creating, including this article’s cover image.)

AI is part of a long history of digital innovation, but it represents a sea change from the technology developments that came before. It has massively accelerated technological innovation—in large part because generative AI tools like ChatGPT are now accessible to anyone with an internet connection. James Vincent at The Verge put it well: “For years, researchers have been on a tear, pumping out new AI models faster than they can be commercialized. But in 2022, a glut of new apps and programs have suddenly made these skills available to a general audience, and in 2023, as we continue scaling this new territory, things will start changing—fast.”

AI has gone mainstream, and the impact is reverberating everywhere—from the halls of Congress to corporate boardrooms. In a Deloitte study of corporate executives conducted at the end of 2020, nearly 80% of executives surveyed said they had implemented some form of AI-driven automation. Another 16% said they planned to do so within three years.

And the data center, of course. According to JLL, “AI requirements, along with continued adoption of cloud services, are the main drivers of hyperscale expansion, driving record growth in the data center sector.” AI is expected to accelerate data center demand even more as cloud providers increasingly offer GPU-as-a-service offerings, AI companies offer new AI services, and enterprises race to adopt the technology.

AI workloads run on high-performance GPUs that are incredibly power intensive. Angus Loten at the Wall Street Journal noted, “Over several days, a single AI model can consume tens of thousands of kilowatt-hours. Generative AI models, such as the technology underlying OpenAI’s ChatGPT chatbot, can be up to 100 times bigger than standard AI tools.” That’s raising data center power requirements and density and forcing changes in how the data center is cooled.

As has always been the case, data center requirements are driven by the capabilities and constraints of the technology infrastructure inside the data center, and the demands of the workloads running on that gear. To support AI workloads, chipmakers are producing increasingly powerful chips. For over a decade, chip performance increased while power consumption remained relatively flat. But since about 2018, rising performance has meant rising power consumption. More computation requires more watts. As just one example, the maximum power consumption of NVIDIA’s latest GPU is 160% higher than that of the company’s previous generation chips.

Today, rising performance means rising power consumption

Source: Omdia research commissioned by Stream Data Centers

 

For the most part, increasingly power-hungry technology infrastructure is being fit into the same data center footprint as previous generations. Average server rack density has been rising steadily. And while the average server deployments remain approximately 10 kW per rack, some more recent deployments are reaching five times that level. Applications such as artificial intelligence run at those much higher densities, so high-density deployments will be the norm in three to five years with the next generation of technology infrastructure.

Supporting more power-hungry workloads, utility power requirements are increasingly massive. By the end of 2021 there were more 20+ megawatt data center requirements than 1 MW requirements, according to JLL. Most data centers being built today are well over 30 MWs. Data centers are still increasingly efficient, but exponential growth in the workloads they support means significant growth in the size of data centers and the amount of electricity they consume in aggregate.

Closed-loop liquid cooling to the rack will be essential
Increasingly massive power requirements and rising density mean more heat transfer rate per unit area (heat flux), which pushes the limits of air-based cooling. We are already seeing that heat fluxes for the most powerful processors are too high to manage with air cooling. Air is not nearly as effective a heat transfer medium as liquid, and at some point is unable to efficiently remove all the heat generated by high-power chips.

Supporting the levels of heat flux that will come with 50+ kW rack densities will require direct liquid cooling to the rack with a direct tie to a chilled water loop via cooling distribution units (CDUs). When it comes to heat transfer, water is fundamentally more efficient than air—because it is denser, has a higher specific heat capacity, and a lower thermal resistance.

Liquid cooling to the rack is also driven by the increasing focus on sustainability beyond PUE. By reducing the amount of energy required and opening more opportunities for economization, liquid cooling to the rack reduces Scope 2 emissions (indirect GHG emissions associated with the purchase of electricity). By reducing the amount of MEP infrastructure required, liquid cooling to the rack reduces Scope 3 emissions (indirect GHG emissions associated with the value chain; essentially, embedded carbon).

How AI has changed data center development
At Stream Data Centers, we’ve been watching these trends develop and have worked hard to prepare ourselves to respond—including through proactive land development, standardized (and configurable) data center design, and flexible deal structures.

Proactive land development
As Data Center Frontier founder and editor-at-large Rich Miller has written, “Huge interest in artificial intelligence is driving a land grab for data center real estate.” At the beginning of the COVID-19 pandemic, Stream saw the emerging trend of high demand and constrained supply and set about expanding our dedicated site development team. Today we control a  large and growing number of properties across North America, all of which have been thoroughly vetted to ensure fast development.

In addition to land constraints, a shortage of available power is inhibiting growth of the data center market, according to CBRE. Because validating utility capacity and managing the deployment of utility infrastructure is among the longest segments of the data center development process, by proactively securing and developing land in key areas (including securing utility power) we’re dramatically cutting time to market for our customers. With an in-house team of deeply experienced experts, proprietary GIS technology, and a systematic and strategic approach, we’re also ensuring the most ideal data center sites.

Standardized and configurable data center design
In 1908, Henry Ford proved the benefits of standardization. You likely know the adage, “Any color the customer wants, as long as it’s black.” Thanks in part to standardization, the Model T was the first widely affordable automobile. But a potential downside of standardization is rigidity, which doesn’t serve well when working with the biggest consumers of data center capacity that have their own designs.

At Stream we balance the benefits of standardization with agility by infusing configurability into our design and construction standard so we can be responsive to customer needs and deliver capacity relatively quickly. For example, a customer could decide to deploy UPS equipment, or not. They could deploy water to the rack, or leverage air cooling. Either way, the decision doesn’t affect the long lead equipment we’ve already procured. Our customers can defer decisions like those until later in the process without extending the development timeline.

Another potential downside of standardization is an inability to pivot quickly. Configurability in our standardized design enables us to support what’s next. For example, seeing the rise of applications leveraging AI and expecting it will raise densities to the level that new cooling approaches will be necessary, we configured our standardized design to easily accommodate air cooling today and liquid cooling to the rack in the future.

Flexible deal structures
Another way we’re responsive to our customers’ particular needs is by offering flexible deal structures, including optionality around ownership. Compared to a publicly traded REIT, real estate developers like Stream typically can offer more flexibility by crafting solutions that meet not just the technical performance specifications but also help meet more traditional ‘business’ goals for ownership, use of capital, etc.

Bottom line: Semper Gumby
Our customers’ technology infrastructure is changing so fast it’s a hard ask for them to lock in a particular decision two years before a new data center is scheduled to be commissioned. Being able to defer decisions, for example on cooling technology, until later gives our customers more freedom to match their data center deployments to their current and future needs. After a data center is operating, rapid technology changes continue to come to bear with each customer refresh cycle.

A data center configurable to support the next generation of technology infrastructure is a data center better suited to support innovation.

About Our Contributor

Mike Licitra

Vice President, Solutions Architecture
As Vice President of Solutions Architecture, Mike Licitra uses his experience at class-leading financial services and cloud organizations to help Stream’s customers align their infrastructure strategy and business goals by optimizing design to maximize performance and minimize total cost of ownership. Read More