SK Hynix outlines shared AI memory pool concept developed with Intel
In brief
- SK Hynix presented a shared memory pool concept at the Intel AI Summit in Seoul.
- Dedicated memory racks serve multiple GPU and CPU servers over CXL and RDMA.
- Time to first token fell from 80.76 to 15.45 seconds in internal tests.
- SK Hynix's own results haven't been independently validated.
How the pooled design works
The idea is fairly simple. Instead of giving every machine its own private stash of memory, SK Hynix wants servers to share a common pool, Crypto Briefing reported. High-capacity memory moves into dedicated racks, and multiple GPU and CPU servers reach it over interconnects like CXL and RDMA. (CXL lets processors treat external memory as if it's nearby, while RDMA lets machines read each other's memory without routing requests through the main processor.) The pooled racks sit alongside existing server setups. They don't replace them.
SK Hynix Vice President Joo Young-pyo pointed to inefficiencies in the traditional approach of attaching high-bandwidth memory directly to each GPU, and he credited progress to joint ecosystem development with partners, according to the outlet.
The KV cache problem
The target is the KV cache. That's the working memory an AI model builds up during a conversation or task, and it grows quickly in multi-turn inference.
In SK Hynix's internal testing, average time to first token using eviction and recompute came in at 80.76 seconds. With remote CXL DRAM from a shared pool, it fell to 15.45 seconds, roughly an 81% reduction.
Those are SK Hynix's own numbers.
They haven't been independently validated, and the company said the architecture will be validated with Intel, customers and other strategic partners starting from the system-design phase. For now it's a concept under development, not a finished joint product.
Building on an existing relationship
The shared pool builds on an existing SK Hynix and Intel relationship. In December 2025, SK Hynix became the first vendor to receive Intel Data Center certification for a 256 GB DDR5 RDIMM on the Xeon 6 platform, per Crypto Briefing. That module delivered up to a 16% increase in inference performance and cut power consumption by approximately 18%, the outlet reported.
The research framing casts Intel CPUs as orchestrators in agentic AI systems, with GPUs handling computation and pooled memory holding context. It also fits SK Hynix's broader focus on memory-centric computing.
Frequently asked questions
What is SK Hynix's shared memory pool?
It's a concept that moves high-capacity memory into dedicated racks that multiple GPU and CPU servers can access over interconnects like CXL and RDMA. It replaces per-machine private memory with a common pool, and the racks are designed to sit alongside existing server setups.
How much faster was AI inference in SK Hynix's tests?
In SK Hynix's internal testing, average time to first token fell from 80.76 seconds using eviction and recompute to 15.45 seconds using remote CXL DRAM from a shared pool, roughly an 81% reduction. These are internal results that haven't been independently validated.
Why does the KV cache matter for AI servers?
The KV cache is the working memory an AI model builds during a conversation or task, and it grows quickly in multi-turn inference. SK Hynix's design targets it as agentic AI workloads raise demand for memory capacity and bandwidth.
