Amazon explains how 28 years of AWS experience shaped its custom chip designs
At the AI Infra Summit on September 16, Amazon senior vice president Peter DeSantis said the company’s 28‑year history with AWS has guided its custom silicon development from Nitro to Graviton to Trainium. He noted that today the vast majority of workloads on AWS run both faster and more cost‑effective on Graviton processors.
The Nitro System moved the virtualization stack onto dedicated hardware, allowing customers to get bare‑metal performance simply by launching an EC2 instance. Graviton processors were developed using internal workload data from Amazon’s own services and customer applications rather than relying on generic benchmarks. Trainium chips were shaped by close work with leading AI customers, and their hardware‑enabled profiler captures fine‑grained telemetry from production workloads without interference. DeSantis highlighted that memory bandwidth often determines how fast a generative model can produce responses, making it a key focus for AI chip design.
“When you're building hardware and building complex systems, understanding the workload deeply is a huge asset,” Peter DeSantis, senior vice president of foundational AI models and custom silicon said.
Amazon now designs chips for networking, general‑purpose computing, and AI, but the company did not set out to become a chip designer. As DeSantis explained, each step in its silicon journey began with a customer workload that needed something better than existing hardware could provide.
What we know
Peter DeSantis told the audience at AI Infra Summit on September 16, each step came from trying to solve a problem customers brought to AWS.
DeSantis has been at Amazon for 28 years and has worked on AWS since EC2 launched 20 years ago.
Around 2016, with the year-over-year gains of Moore's Law slowing, Amazon turned to the general-purpose processor.
Today, DeSantis noted, the vast majority of workloads on AWS run both faster and more cost-effectively on it.
That learning shaped Trainium2 and Trainium3, and the roadmap to Trainium4.