PCIe 8.0 announced by the PCI-Sig will double throughput again

https://www.servethehome.com/pcie-8-0-announced-by-the-pci-sig-will-double-throughput-again/

by rbanffy • 12 months ago

170 222 12 months ago

Any EEs that can comment on at what point do we just flip the architecture over so the GPU pcb is the motherboard and the cpu/memory lives on a PCIe slot? It seems like that would also have some power delivery advantages.

vincheezel• 12 months ago

Good to see I’m not the only person that’s been thinking about this. Wedging gargantuan GPUs onto boards and into cases, sometimes needing support struts even, and pumping hundreds of watts through a power cable makes little sense to me. The CPU, RAM, these should be modules or cards on the GPU. Imagine that! CPU cards might be back..

ksec2• 12 months ago

It is not like CPU aren't getting higher wattage as well. Both AMD and Intel have roadmap for 800W CPU.

At 50-100W for IO, this only leaves 11W per Core on a 64 Core CPU.

avgeek23• 12 months ago

And the memory should be a onboard module on the cpu card intel/amd should replicate what apple did with a unified same ringbus sort of memory modules. Lower latency,higher throughput.

Would push performance further. Although companies like intel would bleed the consumer dry with, a certain i5-whatever cpu with onboard memory of 16 gigs could be insanely priced compared to what you'd pay for addon memory.

sitkack• 11 months ago

https://en.wikipedia.org/wiki/Compute_Express_Link

derefr• 11 months ago

But all of the most-ridiculous hyperscale deployments, where bandwidth + latency most matter, have multiple GPUs per CPU, with the CPU responsible for splitting/packing/scheduling models and inference workloads across its own direct-attached GPUs, providing the network the abstraction of a single GPU with more (NUMA) VRAM than is possible for any single physical GPU to have.

How do you do that, if each GPU expects to be its own backplane? One CPU daughterboard per GPU, and then the CPU daughterboards get SLIed together into one big CPU using NVLink? :P

mensetmanusman• 11 months ago

It’s always going to be a back and forth on how you attach stuff.

Maybe the GPU becomes the motherboard and the CPU plugs into it.

db48x• 11 months ago

No, for a gaming computer what we need is the motherboard and gpu to be side by side. That way the heat sinks for the CPU and GPU have similar amounts of space available.

For other use cases like GPU servers it is better to have many GPUs for every CPU, so plugging a CPU card into the GPU doesn’t make much sense there either.

verall2• 11 months ago

If you look at a any of the nvidia DGX boards it's already pretty close.

PCIe is a standard/commodity so that multiple vendors can compete and customers can save money. But at 8.0 speeds I'm not sure how many vendors will really be supplying, there's already only a few doing serdes this fast...

Melatonic3• 11 months ago

Isnt it about latency as well with DGX boards? Vs PCI-E. You can only fit so much RAM on a board that will realistically be plugging into a slot

eggsome• 11 months ago

Has the DGX actually shipped anywhere yet?

y1n0• 11 months ago

There are companies that specialize in memory controller ip that every one else uses, including large semi companies like Intel.

The ip companies are the first to support new standards, make their money selling to intel etc. Allowing intel or whomever to take their time to build higher performance ip.

bgnn2• 11 months ago

These days you can buy any standard as a soft IP from Synopsys or Cadence. They take their previous serdes and modify it to meet the new standard. They have thousands of employees across the globe just doing that.

MurkyLabs• 11 months ago

Yes I agree, let's bring back the SECC style CPU's from the Pentium Era, I've still got my Pentium II (with MMX technology)

Razengan• 11 months ago

Isn't that what has kinda sorta basically happened with Apple Silicon?

MBCook3• 11 months ago

GPU + CPU on the same die, RAM on the same package.

A total computer all-in-one. Just no interface to the world without the motherboard.

trenchpilgrim• 11 months ago

And AMD Strix Halo.

Dylan168076• 11 months ago

And limit yourself to only one GPU?

Also CPUs are able to make use of more space for memory, both horizontally and vertically.

I don't really see the power delivery advantages, either way you're running a bunch of EPS12V or similar cables around.

burnt-resistor• 11 months ago

Figure out how much RAM, L1-3|4 cache, integer, vector, graphics, and AI horsepower is needed for a use-case ahead-of-time and cram them all into one huge socket with intensive power rails and cooling. The internal RAM bus doesn't have to be DDRn/X either. An integrated northbridge would deliver PCIe, etc.

kvemkon3• 11 months ago

at what point do we just flip the architecture over so the GPU pcb is the motherboard and the cpu/memory

Actually the RapsberryPi (appeared 2012) was based on a SoC with a big and powerful GPU and small weak supporting CPU. The board booted the GPU first.

LeoPanthera2• 11 months ago

Bring back the S100 bus and put literally everything on a card. Your motherboard is just a dumb bus backplane.

MBCook3• 11 months ago

We were moving that way, sorta, with Slot 1 and Slot A.

Then that became unnecessary when L2 cache went on-die.

leoapagano• 11 months ago

One possible advantage of this approach that no one here has mentioned yet is that it would allow us to put RAM on the CPU die (allowing for us to take advantage of the greater memory bandwidth) while also allowing for upgradable RAM.

MBCook3• 11 months ago

Couldn’t we do that today if we wanted to?

What’s keeping Intel/AMD from putting memory on package like Apple does other than cost and possibly consumer demand?

themafia2• 11 months ago

I think you'd want to go the other way.

GPU RAM is high speed and power hungry. So there tends to not be very much of it on the GPU card. This is part of the reason we keep increasing the bandwidth is so the CPU can touch that GPU RAM at the highest speeds.

It makes me wonder though if a NUMA model for the GPU is a better idea. Add more lower power and lower speed RAM onto the GPU card. Then let the CPU preload as much data as is possible onto the card. Then instead of transferring textures through the CPU onto the PCI bus and into the GPU why not just send a DMA request to the GPU and ask it to move it from it's low speed memory to it's high speed memory?

It's a whole new architecture but it seems to get at the actual problems we have in the space.

pshirshov3• 11 months ago

Can I just have a backplane? Pretty please?

colejohnson66• 11 months ago

Sockets (and especially backplanes) are absolutely atrocious for signal integrity.

vFunct• 11 months ago

VMEBus for the win! (now VPX...)

theandrewbailey• 11 months ago

I've wondered why there hasn't been a desktop with a CPU+RAM card that slots into a PCIe x32 slot (if such a thing could exist), or maybe dual x16 slots, and the motherboard could be a dumb backplane that only connected the other slots and distributed power, and probably be much smaller.

guerrilla3• 11 months ago

Yes, for fucks sake, this is the only way forward. It gives us the ultimate freedom to do whatever we want in the future. Just make everything a card on the bus and quit with all this hierarchy nonsense.

dylan604• 11 months ago

Wouldn't that mean an complete mobo replacement to upgrade the GPU? GPU upgrades seem much more rapid and substantial compared to CPU/RAM. Each upgrade would now mean taking out the CPU/RAM and other cards vs just replacing the GPU

p1esk• 11 months ago

GPUs completely dominate the cost of a server, so a GPU upgrade typically means new servers.

chrismorgan• 11 months ago

GPU upgrades seem much more rapid and substantial compared to CPU/RAM.

I feel like I’ve been hearing about people selling five-to-ten-year-old GPUs for sometimes as many dollars as they bought them for, for the last five years; and people choosing to stay on 10-series NVIDIA cards (2016) because the similar-cost RTX 30-, 40- or 50-series was actually worse, because they’d been putting the effort and expense into parts of the chips no one actually used. Dunno, I don’t dGPU.

0manrho7• 11 months ago

We're already there. That's what a lot of people are using DPU's are for.

An example, This is storage instead of GPU's, but as the SSD's were PCIe NVMe, it's pretty nearly the same concept: https://www.servethehome.com/zfs-without-a-server-using-the-...

undersuit• 11 months ago

To continue the ServeAtHome links, https://www.servethehome.com/microchip-adaptec-smartraid-430...

PCI-e Networks and CXL are the future of many platforms... like ISA backplanes.

coherentpony• 11 months ago

The concept exists now. You can "reverse offload" work to the CPU.

bgnn2• 11 months ago

EE here. There's no reason to not deliver power directly to the GPU by using cables. I'm not sure if it's sooving anything.

But you are right, there's no hierarchy in the systems anymore. Why do we even call something a motherboard? There's a bunch of chips interconnected.

j16sdiz• 11 months ago

It is not a EE problem. It is an ecosystem problem. You need a whole catalog of compatible hardware for this.

iszomer3• 11 months ago

I wonder how many additional layers are required in the PCB to achieve this + how this will dramatically affect the TDP; the GPU's aren't the only components with heat tolerance and capacitance.

mcdeltat• 11 months ago

Personally I hope this point comes after we realise we don't need 1kW GPUs doing a whole lot of not much useful

zkms• 11 months ago

My reaction to PCIe gen 8 is essentially "Huh? No, retro data buses are like ISA, PCI, and AGP, right? PCIe Gen 3 and SATA are still pretty new...".

I wonder what modulation order / RF bandwidth they'll be using on the PHY for Gen8. I think Gen7 used 32GHz, which is ridiculously high.

eqvinox4• 11 months ago

I'd highly advise against using GHz here (without further context, at least), a 32Gbaud / 32Gsym/s NRZ signal toggling at full rate is only a 16GHz square wave.

baud seems out of fashion, sym/s is pretty clear & unambiguous.

(And if you're talking channel bandwidth, that needs clarification)

kvemkon3• 11 months ago

> I think Gen7 used 32GHz, which is ridiculously high.
16GHz square wave

Is it for PCIe 5.0? PCIe 6.0 should operate on the same frequency and doubling the bandwidth by using PAM4. If PCIe 7.0 doubled the bandwidth and is still PAM4, what is the underlying frequency?

guerrilla3• 11 months ago

baud seems out of fashion, sym/s is pretty clear & unambiguous.

Huh? Baud is sym/s.

Dylan168076• 11 months ago

PCIe Gen 3 and SATA are still pretty new...

That's an interesting thought to look at. PCIe 3 was a while ago, but SATA was nearly a decade before that.

I wonder what modulation order / RF bandwidth they'll be using on the PHY for Gen8. I think Gen7 used 32GHz, which is ridiculously high.

Wikipedia says it's planned to be PAM4 just like 6 and 7.

Gen 5 and 6 were 32 gigabaud. If 8 is PAM4 it'll be 128 gigabaud...

weinzierl• 11 months ago

Don't forget VESA Local Bus.

bhouston• 11 months ago

I love the PCIe standard is 3 generations ahead of what is actually released. Gen5 is the live version, but the team behind it is so well organized that they have a roadmap of 3 additional versions now. Love it.

ThatMedicIsASpy2• 11 months ago

Gen6 is in use look at Nvidia ConnectX-8

drewg123• 11 months ago

What hosts support Gen6? AFAIK, Gen5 is the most recent standard that's actually deployed. Eg, what can you plug a CX8 into that will link up at Gen6?

tails4e• 11 months ago

It takes a long time to get form standard to silicon, so I bet there are design teams working on pcie7 right now, which won't see products for 2 or more years

Seattle3503• 11 months ago

Is there an advantage of getting so far ahead of implementations? It seems like it would be more difficult to incorporate lessons.

kvemkon3• 11 months ago

When AMD introduces a new Desktop CPU series IIRC they claim the next generation design is (almost) finished (including layout?) and they start with the next-next-gen design. And I'm also asking the same question. But more than a half a year before the CPU becomes available to the public it is already being tested by partners (mainboard manufacturers and ?).

Phelinofist• 11 months ago

So we can skip 6 and 7 and go directly to 8, right?

zamadatix7• 11 months ago

"3 generations" seems like a bit of a stretch. Millions of Blackwell systems use PCIe 6.x today, PCIe 7.x was finalized last month, and this is an announcement work on PCIe 8.0 has started for release in 3 years. I.e. it has only been one month of being one generation behind the latest PCIe revision.

It'll be interesting if consumer devices bother trying to stay with the latest at all anymore. It's already extremely difficult to justify the cost of implementing PCIe 5.0 when it makes almost no difference for consumer use cases. The best consumer use case so far is enthusiasts who want really fast NVMe SSDs in x4 lanes, but 5.0 already gives >10 GB/s for a single drive, even with the limited lane count. It makes very little difference for x16 GPUs, even with the 5090. Things always creep up over time, but the rate at which the consumer space creeps is just so vastly different from what the DC space has been seeing that it seems unreasonable to expect the two to be lockstep anymore.

_zoltan_8• 11 months ago

either I'm misinformed in which case is like to see proof, or you are, but GB200 is not Gen6, just Gen5 and this will only be rectified with GB300 which, while exists in the wild, is not in the millions.

so indeed the parent commenter would be correct that everything is Gen5 right now.

Melatonic3• 11 months ago

CPUs themselves only have so many PCI-E lanes though right? Wouldnt it make sense (even for consumers) to have peripherals using less lanes (but more speed per lane) for a multi GPU system or something that uses a lot of drives?

michaelt2• 11 months ago

"3 generations" seems like a bit of a stretch. Millions of Blackwell systems use PCIe 6.x today

True. And yet, if you buy an RTX 5090 today, costing $2400 and released in January this year, it's PCIe 5.0 x16

jsolson2• 11 months ago

This actually makes sense from a spec perspective if you want to give enough to allow hardware to catch up with the specs and to support true interop.

Contrast this with the wild west that is "Ethernet" where it's extremely common for speeds to track well ahead of specs and where interop is, at best, "exciting."

ThatMedicIsASpy2• 11 months ago

I'll take it if my consumer mb chipset supports giving me 48 PCIe7 lanes if future desktops still would only come with 24 gen 8 lanes

richwater• 11 months ago

Meanwhile paying a premium for a Gen5 motherboard may net you somewhere in the realm of 4% improvements in gaming if you're lucky.

Obviously PCI is not just about gaming but...

simoncion2• 11 months ago

From what I've seen, the faster PCI-E bus is important when you need to shuffle things in and out of VRAM. In a video game, the faster bus reduces the duration of stutters caused by pushing more data into the graphics card.

If you're using a new video card with only 8GB of onboard RAM and are turning on all the heavily-advertised bells and whistles on new games, you're going to be running out of VRAM very, very frequently. The faster bus isn't really important for higher frame rate, it makes the worst-case situations less bad.

I get the impression that many reviewers aren't equipped to do the sort of review that asks questions like "What's the intensity and frequency of the stuttering in the game?" because that's a bit harder than just looking at average, peak, and 90% frame rates. The question "How often do textures load at reduced resolution, or not at all?" probably requires a human in the loop to look at the rendered output to notice those sorts of errors... which is time consuming, attention-demanding work.

Dylan168076• 11 months ago

There's a good amount of reviewers showing 1% lows and 0.1% lows, which should capture stuttering pretty well.

I don't know how many games are even capable of using lower resolutions to avoid stutter. I'd be interested in an analysis.

rbanffy5• 11 months ago

I’m sure Windows performance counters can track the volume of data going between CPU memory and VRAM over the PCIe bus.

jeffbee• 11 months ago

By an overwhelming margin, most computers are not in gamers' basements.

checker659• 11 months ago

No matter the leaps in bandwidth, the latency remains the same. Also, with PCIe switches used in AI servers, the latency (and jitter) is even pronounced.

LeoPanthera2• 11 months ago

I thought we were only just up to 5? Did we skip 6 and 7?

pkaye• 11 months ago

Some of the newer ones maybe more for data centers.

robotnikman• 11 months ago

I know very little about electronics design, so I always find it amazing that they keep managing to double PCIe throughput over and over. Its also probably the longest lived expansion bus at the moment.

wmf4• 11 months ago

It's less surprising if you realize that PCIe is behind Ethernet (per lane).

extraduder_ire• 11 months ago

Which Ethernet standard? Light online searching leads me to believe it tops out at 800Gb/s. (and not in a format I would normally call Ethernet)

rbanffy5• 11 months ago

I’m sure you can get some VMEbus boards.

pshirshov3• 11 months ago

Yeah, such a shame I've just upgraded to a 7.0 motherboard for my socket AM7 CPU.

Being less sarcastic, I would ask if 6.0 mobos are on the horizon.

wmf4• 11 months ago

I guess Venice, Diamond Rapids, and Vera will have 6.0.

_zoltan_8• 11 months ago

GB300 has Gen6 so I am pretty sure GV200 would have it at least.

I'm fairly sure they are cooking Gen7 already into CX9.

_zoltan_8• 11 months ago

what I don't get: why doesn't AMD just roll Gen6 out in their CPU, bifurcate it to Gen5, and boom, you have 48x2 Gen5s? same argument for gen5 bifurcated to gen4.

this would solve the biggest issue with non-server motherboards: not enough PCIe lanes.

zamadatix7• 11 months ago

Bifurcation can't create new lanes, only split the existing lane count up into separate logical slots.

What you're saying is possible though, you just need something a little heavier like a PCIe switch do the lane count + mixed-speed conversion magic. That's exactly what the chipset is, but for various reasons it's still only PCIe 4.0, even on the latest generation chips. I wouldn't be surprised if that changed again next generation. The downsides of this approach are switches add cost, latency, and can consume a lot of power. When they first upped the chipset to be a PCIe 4.0 connection in the 500 era, most motherboards capable of using the bandwidth of the chipset actually had to add chipset cooling.

Ideally they'd just add alternative options with more lanes directly from the CPU, but that'd add competition to the bottom of the Threadripper line.

michaelt2• 11 months ago

this would solve the biggest issue with non-server motherboards: not enough PCIe lanes.

Is that a problem?

These days, you don't need slots for your sound card and network card, that stuff's all integrated on the motherboard.

Plenty of enthusiast motherboards support 1 GPU, 3-4 nvme drives, 4 SATA drives, and have a PCIe 1x slot or two to spare.

Is anyone struggling, except folks working with LLMs? Seems to me folks looking to put several $2400 RTX 4090s in one machine ain't exactly a big market. And they'll probably want a giant server board, so they have space for all their giant four-slot-wide cards.

mrene• 11 months ago

Devices integrated on the motherboard will either connect to the cpu via PCIe or USB, lanes aren't just for PCIe cards.

For example, motherboards with multiple nvme drives often have 1 drive with dedicated lanes and the remainder multiplexed through a PCIe switch embedded in the chipset.

Melatonic3• 11 months ago

I feel like what we really need is a GPU "socket" like we have for CPU's. And then a set of RAM slots dedicated to that GPU socket (or unified RAM shared between CPU and GPU)

Sephr• 11 months ago

HBM would most likely have to be integrated directly on the GPU module given performance demands and signal constraints.

Lramseyer• 11 months ago

Unfortunately socketed processors only really work with DDRx type DRAM interfaces. GPUs use GDDR and HBM interfaces, which are not ideal for sockets. In the case of HBM, you have 1024 data traces per DRAM chip, which would make the socket have an insane number of pins. GDDR has fewer pins, but makes up for it with higher data rates (32 pins at 16Gb/s) and that is impractical to use in a socket due to the variance in contact area resulting in impedance matching issues.

bpbp-mango• 11 months ago

I wonder if this will help applications like VPP/DPDK. not sure if the CPU or the lanes are the bottleneck there.

iFred• 11 months ago

I can't be the only one let down that there wasn't some new slot design. Something with pizzazz and flare.

top_sigrid• 11 months ago

For all the others seeing the HN Hugg of death :)

https://archive.is/oa81K

0xffany• 11 months ago

Crazy to see STH get the HN hug of death!

One would think they get sizeable traffic as-is.

rfl890• 11 months ago

So I can run a full powered PCIe 4.0 16x GPU on 1 PCIe 8.0 lane???

PCIe 8.0 announced by the PCI-Sig will double throughput again

Related Stories