When users query a generative AI engine such as ChatGPT, the responses come quickly. The technology relies on an incredible amount of data to produce answers and even generate new data, but it works so seamlessly that there’s hardly a hint at the massive infrastructure churning in the background.
In truth, it is this infrastructure behind the scenes that makes all the difference. Hyperscale data centers are tasked with enabling a new AI-powered world, but as usage grows, the job becomes more difficult. AI models are becoming increasingly sophisticated and interactive, and the data that trains them vaster, encompassing not only text but multimodal content like audio, video and graphics. Many of the most useful AI models take months to train. As AI demand grows, it’s vital that data centers find ways to rapidly increase capacity and reduce training times.
To keep up, they’re turning to innovative solutions from companies like Astera Labs (NASDAQ: ALAB), which is on a mission to unleash the full potential of AI and cloud infrastructure.
“Combining processing advances and interactive models with richer forms of data is what is driving all AI use cases forward,” says Thad Omura, chief business officer at Astera Labs. “It’s setting up an industry backdrop for what seems to be an unlimited amount of compute power required.”
The connectivity challenges posed by AI’s growth
Just a short time ago, it seemed like only a few people had even heard of generative AI. By early 2024, according to McKinsey’s State of AI report, 65% of organizations were regularly using it. The growth has rivaled some of history’s most influential technologies, and it doesn’t appear to be slowing.
Currently valued at $196.6 billion, the AI industry is expected to grow by 36.6% annually through 2030, Grand View Research says. Omura says that in the last year, there have been several massive AI infrastructure buildouts. What comes next is a wave of applications that put that infrastructure to use.
“We’re in the early innings of reliable and efficient buildouts,” Omura says of the rollout of AI technology.
Building the compute power to support AI systems will be tricky, as AI requires an interconnected web of powerful GPUs, AI accelerators, CPUs, networking devices, memory and storage. “In a traditional computer architecture, you have a compute element that talks to an endpoint,” Omura says. “In AI infrastructure, you have all these parallel compute devices that all talk to each other at the same time. Reliability is critical; one broken link can degrade the entire system and force a time-consuming AI training to restart from the beginning.”
“Combining processing advances and interactive models with richer forms of data is what is driving AI use cases forward. It’s setting up an industry backdrop for what seems to be an unlimited amount of compute power required.”
— Thad Omura, Astera Labs
It’s only by improving reliability and optimizing the underlying components that we’ll maximize AI’s performance and scale. “In some ways, we think connectivity actually has to outpace (AI compute growth) to make the scalability of all this infrastructure work optimally,” Omura says.
Data centers face three key challenges. The first boils down to reach. AI models consume so much information that the required physical components are sprawled across data centers the size of football stadiums. There are limits on how fast a signal can traverse those distances. Second, as AI use cases become more varied, data centers are introducing an even more diverse range of components, which increases the overall connectivity complexity.
The final problem is one of efficiency. Hyperscalers have deployed billions of dollars in AI infrastructure, so it’s critical that they protect their investment by ensuring the fleet runs at maximum performance—and that they can quickly identify and mitigate issues as they arise.
Improving connectivity to realize AI’s full potential
As the demand for AI-optimized connectivity grows, Astera Labs is helping hyperscalers enhance their operations through its Intelligent Connectivity Platform. The platform combines two main components—silicon-based hardware solutions and a layer of software that gives data center operators a clear, customized view of their AI infrastructure health.
The hardware works to solve the challenge of high-speed signal reach over distance. By enhancing the way different infrastructure components communicate with each other, Astera Labs improves the speed and reliability of compute. Omura likens it to the difference between a single sprinter and a relay team. “Athletes get tired and may not be able to run the entire distance,” Omura says. “You add Astera Labs into the infrastructure, it’s like adding one or many team members to your relay race.”
And then there’s the software component. Customers are able to configure and manage Astera Labs’ silicon-based hardware solutions through a software solution called COSMOS (COnnectivity System Management and Optimization Software). With the COSMOS suite, Astera Labs can help monitor and diagnose connectivity issues in data centers. Astera Labs ensures compatibility of its solutions with its customers’ systems through regular interoperability testing, while COSMOS enables fine-tuning of its solutions as deployed in those systems.
The company’s aim is a connectivity environment that will not only improve how AI operates today but be better prepared to handle massive growth in the future.
“Our goal is to interconnect it all as high performant and reliably as possible,” Omura says. “That is where we make the most impact to the deployment of AI infrastructure across the industry.”