Redesigning AI hardware for a low-power future

As AI models scale into data centers, industrial systems, chatbots, and autonomous agents, computational demand is exploding. Yet the limiting factor is no longer just compute power or algorithms — it is how efficiently data can be moved and processed inside hardware. Every data transfer costs energy. At imec’s ITF World 2026 in Antwerp, Samsung Electronics CTO Jaihyuk Song outlined how the company is redesigning its AI hardware stack.

AI is now deployed across almost every domain. As a result, computational demand is rising at an extraordinary pace. However, this growth is hitting a fundamental limit. The bottleneck is mainly the way data is moved and processed within hardware systems

Memory: a key role

Memory plays a central role. One of the most important enabling technologies is HBM, or High Bandwidth Memory. This type of memory is designed for extremely fast data processing, particularly in AI accelerators and GPUs. Instead of relying on a single memory layer, HBM stacks multiple memory chips vertically. This allows many more data pathways to operate simultaneously, delivering significantly higher bandwidth than conventional memory architectures. Samsung has already been advancing toward HBM4 and HBM4E, the latest generations. “HBM4 has more than double the bandwidth of HBM3, while also delivering significantly improved power efficiency”, Song adds.

Samsung is also working on custom HBM solutions tailored for specific AI workloads. One example is cHBM, a design approach aimed at reducing internal data movement within AI systems. Song: Samsung cHBM minimizes data movement and maximizes power efficiency, “delivering 2.8× higher power-normalized performance compared to a conventional CPU.”

In addition, Samsung is exploring next-generation technologies such as zHBM and advanced packaging solutions using optical signaling. zHBM focuses on more advanced forms of chip integration, where wafer-to-wafer bonding plays a more significant role. In parallel, optical interconnects aim to move data not only through electrical pathways but also through light, enabling higher-speed communication inside data centers. Samsung has not yet provided a concrete timeline for zHBM.

No more microbumps

One of the most critical improvements in HBM comes from how the stacked layers are connected. The traditional method is Thermal Compression Bonding (TCB) to link chips. While effective, this approach introduces physical distance between layers, increases electrical resistance, and makes heat dissipation more difficult.

Hybrid Copper Bonding (HCB) addresses these limitations. In this approach, most microbumps are removed, and the layers are directly connected through copper-to-copper bonding. This creates a much tighter and more efficient connection between memory layers. As a result, thermal performance improves because heat can be dissipated more effectively. At the same time, energy consumption decreases, and chips can be stacked more densely without overheating. In simple terms, it enables more performance in the same space, with less energy loss. Song adds: “Testing showed that HCB can reduce interconnect resistance by up to 18%,” improving both signal efficiency and thermal performance.

However, the company has not yet confirmed when HCB will be introduced into commercial HBM products. The real impact of these technologies on HBM performance and AI workloads will become fully clear once they move from prototypes into large-scale production.

Conclusion: an integrated AI hardware stack

Overall, the combination of these advancements shows the direction Samsung is heading: a fully integrated AI hardware stack designed to move data faster, more efficiently, and with far less energy loss.