Enterprises need to think beyond GPUs for agentic AI, analysts say

The ongoing shift from generative AI (genAI) to agentic AI provides an opportunity for enterprises to move to more nimble and less expensive forms of computing, according to analysts. Early AI models were largely built on expensive GPUs from Nvidia and AMD that offered raw processing power. But newer agentic AI tools, rooted in business process and workflow management, can run on more efficient, cost-effective hardware. As a result, IT decision-makers who still think they require GPUs for anything AI-related need to reconsider their hardware options in terms of both cost and capabilities, analysts said. “A better way of thinking about this is the cost of AI compute and now agentic AI platform services or systems,” said Leonard Lee , principal analyst at Next Curve. “’AI computing’ or ‘accelerated computing’ has clearly transcended the GPU as an inference accelerator.” The new hardware options include CPUs and specialized AI chips, also known as ASICs in semiconductor parlance. Although these chips have been around for years, they are now showing real utility as agentic AI goes mainstream. For one, the CPU — the main chip in any computer — is seeing something of a revival. “The CPU is reinserting itself as the indispensable foundation of the AI era. The CPU now serves as the orchestration layer and critical control plane for the entire AI stack,” Lee said. CPUs are both power efficient and well-suited for AI on the edge, although specialized low-power chips are more capable depending on the task, said Jim McGregor , principal analyst at Tirias Research. “It will still be more efficient to use an ASIC instead of a CPU, and in most cases it will be less expensive over the life of a platform,” he said. The growth of inference provides an opening for optimized AI accelerators, which can handle those jobs more efficiently than GPUs, said Mike Feibus, principal analyst at FeibusTech. “…The relative importance of [the] CPU is rising.” Nvidia — sensing that it needed a low-power chip beyond its power-hungry GPUs — has already introduced an ASIC for inferencing in its hardware stack. And it recently licensed AI chip technology from Groq for $20 billion . Because Agentic AI involves a different computing model than genAI training on GPUs , enterprises need to consider the hardware options and pricing models available through cloud providers. “It’s more about model management than about model building — and the CPU is critical in providing workflow management,” said Jack Gold , principal analyst at J. Gold Associates. Pricing variations continue to be an issue. Straight CPU compute is not billed the same as heavy GPU use, making it difficult to nail down costs, Gold said. “GPUs in training use more electricity generically due to near 100% utilization in a training workload, whereas in general-purpose compute, servers and CPUs run more like 40% to 60% utilization,” he said. “But it’s highly variable depending on what the agent is doing.” Gold predicts that 80% to 85% of AI workloads will move to inference in the next two to three years, especially as tools become more agentic. “CPUs take on a major significance in making everything work. It’s why all the hyperscalers are now loading up on CPUs, not just GPUs,” Gold said. Major cloud providers Google, Amazon and Microsoft , for instance, have their own CPUs and low-power ASICs for inferencing. What looks at the moment like a resurgence in CPU demand is actually pointing to a larger issue: the growing complexity of AI infrastructure, said Gaurav Shah , vice president of business development and strategic partnerships at NeuReality. The overhead around data movement, orchestration and networking is exploding, Shah said. “That’s what’s driving demand — not CPUs doing more AI, but systems struggling to keep up with AI,” Shah said. Beyond enterprises, genAI companies, AI-native companies and neoclouds all will need to rethink their architecture. “The winners will be the architectures that deliver the most inference per watt, not the most cores per server,” Shah said.

Go to News Site