I still don't understand why graphics cards haven't evolved to include sodimm slots so that the vram can be upgraded by the end user. At this point memory requirements vary so much from gamer to scientist so it would make more sense to offer compute packages with user-supplied memory.
tl;dr GPU's need to transition from being add-in cards to being a sibling motherboard. A sisterboard? Not a daughter board.
One of the reasons GPUs can have multiples of CPU bandwidth is they avoid the difficulties of pluggable dimms - direct soldered can have much higher frequencies at lower power.
It's one of the reasons why ARM Macbooks get great performance/watt, memory being even "closer" than mainboard soldered RAM so getting more of those benefits, though naturally less flexibility.
Even DDR5 has this problem. Go look at what soldered DDR5 can do frequency wise compared to DIMMs. It's one of the problems the new CAMM form factor aims to help solve, making it tractable to push the memory frequency up beyond what DIMMs can get yout currently.
I have always wondered: would it be possible to put memory on the back side of the motherboard to get I closer to the CPU? And if it is, would it solve anything else than ram clearance for CPU coolers?
This makes sense. Do we need to begin exploring different socket configurations? IE, using something that more closely resembles a CPU socket versus a traditional RAM slot.
tl;dr GPU's need to transition from being add-in cards to being a sibling motherboard. A sisterboard? Not a daughter board.