Nvidia's homegrown memory design is almost standardized and ready for everyone to use — JEDEC says SOCAMM2 compact DRAM module for AI servers boasts higher speeds and broader compatibility
Nvidia-backed SOCAMM needed more refinement to be production-ready
SOCAMM2 is on track to become the definitive version of the SOCAMM standard very soon. JEDEC has announced that SOCAMM2's design is nearing completion and will take advantage of the same LPDDR5X memory type as its predecessor. Two upgrades will be added to SOCAMM2: an SPD profile (what most might refer to as a "JEDEC profile") and improved LPDDR5X transfer rates of up to 9600 MT/s.
The new memory type will be incorporated into next-gen AI servers, says JEDEC. Although no official announcements have been made, Nvidia seems likely to take another shot at introducing SOCAMM2 alongside its Vera CPUs on its next-generation Rubin platform. Servers integrating the first SOCAMM memory modules were reportedly abandoned by Nvidia due to technical difficulties.
The multi-trillion-dollar AI giant reportedly couldn't get SOCAMM to operate within temperature limits, causing overheating on its Blackwell Ultra GB300 "Cordelia" platform. As such, Nvidia reconfigured "Cordelia" to traditional LPDDR memory for its release.
For reference, SOCAMM is a new memory design that is aimed squarely at the data center, particularly AI servers. SOCAMM is a more compact version of traditional DRAM DIMMs that purports to be more power-efficient.
For example, Micron's SOCAMM 128GB module allegedly consumes just a third of the power of an equivalent 128GB RDIMM. DRAM power consumption is very important in data center environments, where memory power consumption can often exceed that of the CPUs it's attached to.
The new form factor was inspired by CAMM2, and both SOCAMM and CAMM focus on improving memory capacity per unit area compared to traditional DIMMs. For example, the original SOCAMM modules from Micron measured just 14x90mm while featuring four 16-die LPDDR5 memory stacks.
Nvidia's original version was completely proprietary and was only compatible with its own hardware. It also only supported LPDDR5X transfer speeds of up to 8533 MT/s. SOCAMM2 supports even faster LPDDR5X and, in collaboration with JEDEC, should be available for hardware manufacturers other than Nvidia to adopt.
Get Tom's Hardware's best news and in-depth reviews, straight to your inbox.
There's no official release date for SOCAMM2, but it is likely that the design will be finished in the next few months. This will give SK Hynix, Samsung, and Micron time to build production SOCAMM2 modules and, critically, give Nvidia time to test the modules for its Rubin deployments beginning next year.
Follow Tom's Hardware on Google News, or add us as a preferred source, to get our latest news, analysis, & reviews in your feeds.
Aaron Klotz is a contributing writer for Tom’s Hardware, covering news related to computer hardware such as CPUs, and graphics cards.
-
edzieba ReplySOCAMM is a more compact version of traditional DRAM DIMMs that purports to be more power-efficient.
And more importantly, it achieves that lower power at higher bandwidths, as it avoids all the signal integrity issues of DIMM slots and gets close to the performance of DRAM dies directly soldered to the host PCB.
The improvement in packaging volume over right-angle SODIMM slots is a minor factor at most. -
bit_user Reply
What I think SOCAMM/2 does right that spoiled the prior CAMM designs is the fact that it's single-sided. Heat is becoming a significant issue for DDR5 scaling, hence why I think it was a bad decision for those other designs to put DRAM chips on the underside, where they'd be sandwiched against the motherboard.edzieba said:The improvement in packaging volume over right-angle SODIMM slots is a minor factor at most. -
Elrabin Reply
It's less about the volume gain vs traditional RDIMMs and more the gains in reduction in Z-height.edzieba said:And more importantly, it achieves that lower power at higher bandwidths, as it avoids all the signal integrity issues of DIMM slots and gets close to the performance of DRAM dies directly soldered to the host PCB.
The improvement in packaging volume over right-angle SODIMM slots is a minor factor at most.
The SOCAMM/2s being "flat" to the board means you have more overhead clearance for ever larger heatsinks/coldplates for air/liquid cooling. You want the memory as physically close to the socket as possible, but not encroach upon the cooling method. This achieves both.
With Nvidia scaling to 4400 watts per GPU in only a few years, they're trying to maximize cooling any way they can. From larger coldplates to microchannel etching. -
thestryker Reply
Every LPDDR CAMM module is single sided. This design is literally just to do away with the tab which makes LPCAMM modules wider.bit_user said:What I think SOCAMM/2 does right that spoiled the prior CAMM designs is the fact that it's single-sided. Heat is becoming a significant issue for DDR5 scaling, hence why I think it was a bad decision for those other designs to put DRAM chips on the underside, where they'd be sandwiched against the motherboard. -
thestryker Reply
You cannot compare LPDDR to DDR like you're doing here. They're using LPDDR due to density of both capacity and bandwidth. I'm actually a little surprised they moved away from using soldered memory, but I imagine this allows for simpler speed and capacity differentiation (it may also help with board manufacture over time).edzieba said:And more importantly, it achieves that lower power at higher bandwidths, as it avoids all the signal integrity issues of DIMM slots and gets close to the performance of DRAM dies directly soldered to the host PCB. -
bit_user Reply
Okay, you're right. But, it took me like a dozen tries to find an article that showed the bottom side. It's like memory makers don't want to show us that side, and every tech article just reused the images they provided.thestryker said:Every LPDDR CAMM module is single sided. This design is literally just to do away with the tab which makes LPCAMM modules wider.
Here's one that shows a diagram of a LPCAMM's bottom side:
https://www.cnx-software.com/2023/09/29/samsung-lpcamm-integrates-lpddr-ram-on-removable-modules/
I did find a couple that showed the bottom of non-LP CAMMs, and those definitely have chips on the bottom side. -
bit_user Reply
Also, memory degrades with use. Having field-replaceable memory could be a win for reducing ownership costs, since it saves you having to replace the entire module (which includes the processor die).thestryker said:I'm actually a little surprised they moved away from using soldered memory, but I imagine this allows for simpler speed and capacity differentiation (it may also help with board manufacture over time). -
thestryker Reply
Yeah I couldn't find pictures of the bottom either which was one of the reasons I registered at JEDEC so I could download the specs.bit_user said:Okay, you're right. But, it took me like a dozen tries to find an article that showed the bottom side. It's like memory makers don't want to show us that side, and every tech article just reused the images they provided.
Yeah those it depends on the type a very common one I've seen is 12 chips on top 4 on the bottom. I still think 16 on top is the likely consumer one should that day arrive even though that means a 64GB limit.bit_user said:I did find a couple that showed the bottom of non-LP CAMMs, and those definitely have chips on the bottom side.