This article introduces a 192-Gb 896-GB/s 12-high stacked third-generation high-bandwidth memory (HBM3 DRAM) with low power consumption and high-reliability traits. New design schemes and features, ...including internal low-voltage signaling, center strobe calibration, through-silicon via (TSV) auto-calibration, a symbol-correcting in-DRAM ECC, and machine-learning-based layout optimization, allow large amounts of data transfers among the vertically stacked base and core dies with limited delay mismatch or SI degradation, as well as reduced power consumption from low-voltage swings. Experimental results confirm 896-GB/s bandwidth operations at 1.0-V voltage conditions with up to 15% improved power efficiency.
Ever since the introduction of high bandwidth memory (HBM DRAM) and its succeeding line-ups, HBM DRAM has been heralded as a prominent solution to tackle the memory wall problem. However, despite ...continual memory advancements the advent of high-end systems, including supercomputers, hyper-scale data centers and machine learning accelerators, are expediting requirements for higher-performance memory solutions. To accommodate the increasing system-level demands, we introduce HBM3 DRAM, which employs multiple new features and design schemes. Techniques such as an on-die ECC engine, internal NN-DFE I/O signaling, TSV auto-calibration, and layout optimization based on machine-learning algorithms are implemented to efficiently control timing skew margins and SI degradation trade-offs. Furthermore, reduced voltage swings allow for improved memory bandwidth, density, power efficiency and reliability.
With the emergence of large-language models (LLM) and generative AI, which require an enormous amount of model parameters, the required memory bandwidth and capacity for high-end systems is on an ...unprecedented increase. To meet this need, we present an extended version of the high-bandwidth memory-3 (HBM3 DRAM), HBM3E, which achieves a 1280GB/s bandwidth with a cube density of 48GB. New design schemes and features, such as all-around power-through-silicon via (TSV), a 6-phase read-data-strobe (RDQS) scheme, a byte-mapping swap scheme, and a voltage-drift compensator for write data strobe (WDQS), are implemented to achieve extended bandwidth and capacity with enhanced reliability. The overall architecture and specifications, such as bump map footprint, the number of channel and I/Os, and the operation voltage, are identical to the latest HBM3 1, 2; therefore, backward compatibility is provided, avoiding system modification.
There is enormous demand for high-bandwidth DRAM: in application such as HPC, graphics, high-end server and artificial intelligence. HBM DRAM was developed 1 using the advances in package technology: ...TSV, microbump and silicon-interposer. Owing to these advances, HBM has a much higher bandwidth, at a lower pin speed rate, than conventional DRAM. However, the 3D-stack structure causes TSV interface and PDN problems: TSV connection failure and 3D-accumulation of IR drop, which increases the total cost of HBM. Moreover, as memory bandwidth increases DRAM architectural challenges arise, power consumption and associated thermal problems increase as well.
