According to industry sources, Groq plans to integrate three-dimensional stacked DRAM into its next Language Processing Units to increase memory capacity and bandwidth without sacrificing its hardware’s high throughput. The current Groq 3 LPU includes 500 MB of SRAM, delivering 150 TB/s of bandwidth, approximately seven times the HBM bandwidth of comparable GPUs, and allows the Groq 3 LPX system to generate up to about 3,400 tokens per second. However, a 256-chip rack provides only 128 GB of SRAM in total, forcing operators of trillion-parameter models to distribute model weights across multiple chips. Groq favors standard interfaces and through-silicon interconnections to limit manufacturing risks, in a similar approach to Cerebras’ work. In December 2025, Nvidia reached an approximately $20 billion deal covering the licensing of Groq’s intellectual property and the hiring of its key personnel, an arrangement the research characterizes as effectively an acquisition.
Source: Read the original article
