Artificial intelligence is pushing existing memory technologies to their limits. As large language models continue to grow, they require enormous amounts of memory to store billions or even trillions of parameters. An IEEE Spectrum article explores an emerging technology called High Bandwidth Flash (HBF), which could help address this challenge by combining the high capacity of NAND flash with much higher data transfer speeds than conventional storage devices.
HBF builds on the same concept that made High Bandwidth Memory (HBM) successful. Instead of using stacked DRAM chips, HBF vertically stacks NAND flash dies using advanced 3D packaging techniques. Although traditional flash memory is known for slow write performance, it can achieve much faster read speeds when optimized. Because AI inference primarily reads model weights rather than constantly rewriting them, HBF is particularly well suited for this workload.
The technology is not intended to replace HBM. Instead, it would serve as a complementary memory tier. HBM would continue handling the most time-sensitive operations, while HBF would store massive, largely static datasets such as AI model weights and key-value caches. This approach could free valuable HBM capacity, reduce the number of accelerators required to run large models, and improve the overall efficiency of AI systems.
SanDisk has outlined an ambitious roadmap for the technology. Its first-generation HBF is expected to stack up to 16 NAND flash chips, providing as much as 512 GB of capacity per stack with projected read bandwidth reaching 1.6 TB per second. Future generations aim to increase bandwidth even further, bringing flash closer to the performance needed for demanding AI applications.
The article also notes that HBF remains in its early stages. SanDisk and SK Hynix have launched a joint effort within the Open Compute Project to develop an industry standard, but commercial deployment is still several years away. If successful, HBF could lower costs, improve energy efficiency, and help data centers scale AI inference more effectively without depending exclusively on increasingly scarce and expensive HBM. Rather than replacing existing memory technologies, High Bandwidth Flash may become an essential addition to future AI hardware architectures.
