Datacenter Interconnects: Copper vs. Optics, Nvidia's 78-Layer PCB, Co-Packaged Optics (CPO)

Vik Sekar and Austin Lyons tackle the biggest bottleneck in inference: moving data. They break down the three tiers of datacenter networking — scale up, scale out, scale across — and the core engineering trade-off at each layer: copper vs. optics. Topics include Nvidia's extreme measures to keep scale-up fabric electrical (a 78-layer mid-plane PCB), why Co-Packaged Optics is the "holy grail" everyone wants and no one can ship, and the serviceability problem standing in its way.

Key Takeaways:

• A single 72-GPU scale-up rack needs 5,000+ cables spanning ~2 km — at that density, the power and cost of every connection becomes a system-level constraint, not an implementation detail.
• The scale-up rule is "copper when you can, optics when you must": each pluggable optical module adds ~30W, and with thousands of links in the fabric, that penalty compounds fast.
• Nvidia is pushing copper past its usual limits with a 78-layer mid-plane PCB — 3x+ the layer count of a typical complex board — specifically to avoid paying the optics power tax in its scale-up fabric.
• Scale-up isn't just a training problem. Frontier MoE models need 72-GPU domains to hit memory bandwidth targets, which pulls high-performance interconnect into the inference conversation.
• Scale-up has the highest connection density of the three tiers, making it the largest TAM and the sharpest three-way fight between NVLink, UA-Link, and Ethernet.
• Co-Packaged Optics could cut interconnect power by two-thirds — but a single failed laser could brick an entire multi-thousand-dollar GPU package, and that serviceability risk is what's keeping it on the roadmap instead of in racks.

Chapters:
0:00 The Biggest Problem in Computing
7:14 The Three Tiers of Networking
13:49 Scale Up: Copper vs. Optics
17:59 Front-End vs. Back-End Networks
22:12 The Physical Scale of Cabling
28:46 Nvidia's 78-Layer Mid-Plane
33:16 How Optical Transceivers Work
39:16 The Power Penalty of Pluggables
42:09 The Business of Speed Transitions
44:40 The Promise and Peril of CPO
51:45 The Holy Grail of Networking

Follow Chipstrat:
Newsletter: https://www.chipstrat.com
X: https://x.com/austinsemis

Follow Vik:
Newsletter: https://www.viksnewsletter.com/
X: https://x.com/vikramskr

Follow Semi Doped:
Get more of Austin and Vik daily, free: https://daily.semidoped.com/