USB-C Hubs and Ethernet (2020)

https://lucumr.pocoo.org/2020/7/6/usb-c-network-hubs/

by defaulty • 5 years ago

45 34 5 years ago

lucumr.pocoo.org

I have USB 3.0 and 3.1 hubs; I am not surprised about this article. USB 3.0/3.1 hubs are fickle as heck! And I noticed that they seem to emit some kind of EMF around the port which means they are unshielded. It rendered some USB dongle non-functional until I unplug the hub.

Now I have Anker 3.1 Mini-Dock (without the ethernet), it works well but it has issues with HDMI. For whatever the reason, the system think that the HDMI is plugged in the hub whereas it is not. So, It cause a strange issue with multimonitor since the system believes that there is a second monitor plugged to it. But yet it couldn’t show the second monitor in the display setting but it insists that it is plugged.

asdff• 5 years ago

I don't know if it was just my hub or the crappy router that Spectrum gives me, but whenever I used the ethernet on my hub with my laptop, it killed the ethernet connection to the desktop.

cbhl• 5 years ago

I've had good luck with a Thunderbolt 4 Hub from Anker as of late. The hub has a power button of its own, and when the hub is turned off it turns off the attached USB-C-to-Ethernet adapter.

recursivedoubts• 5 years ago

over under on when USB-C will finally "just work"?

five years ago I would have said five years

now I'd say 10

wyager• 5 years ago

2 months before USB-D hits consumer devices.

senectus1• 5 years ago

https://www.datapro.net/news/usb-type-d-connector-unveiled.h...

oneplane4• 5 years ago

This has very little to do with USB-C. The same problem can be observed using a classic USB-A to ethernet adapter when connected to a class USB-A hub that has its own power supply.

The problem is the 'feature' of some USB Ethernet controllers where they keep the ethernet link active when the host disconnects. As described in the article, it's the PAUSE frame that some switches forward to all devices on the network causing them to... pause sending data. The link stays up but nothing happens. Perhaps the idea was that you get a fast connection resume, but it's not like ethernet handshaking takes a lot of time so they might as well simply disconnect the ethernet link when the USB ethernet controller notices the host computer has disconnected.

sannee• 5 years ago

Perhaps the idea was that you get a fast connection resume

This seems unlikely for the reasons you listed. Since USB is a polled bus and has shared bandwidth, implementing the flow control is likely very beneficial in case the bus can't temporarily handle the full Ethernet speed.

So, a USB-to-Ethernet gizmo just monitors its internal Ethernet-to-USB queue and if it starts to overflow, it attempts to flow control the switch. But, if no USB host is connected, no USB data requests ever arrive, the queue overflows and never gets cleared...

There may also be some WoL/USB remote wakeup stuff at play, which may be why these ICs do not drop the link on USB host disconnect by default. In fact, I wonder if messing with those settings on the host side could remediate the issue...

SAI_Peregrinus• 5 years ago

No, the problem is some broken switches forwarding PAUSE frames to other ports, in violation of the 802.3 spec. It's perfectly compliant for the adapter to send PAUSE frames when it can't empty the incoming packet queue, but that should only pause the single port on the switch.

markstos• 5 years ago

tl;dr: cheap hubs with ethernet failed, dedicated Apple ethernet adapter worked. Author still isn't sure exactly why.

procarch20192• 5 years ago

TBF, hubs in general seem to fail regardless of brand. See my other comment about Dell hubs vs docking stations. A model of ‘hubs are cheap because they fail frequently’ is a poor one. Hubs should be cheap because they can’t deliver power and device connections or they have limited port types.

jscheel• 5 years ago

I have been consistently blown away at just how bad usb hubs really are. The market is absolutely saturated with garbage, but it seems like even the high quality ones aren't really that good.

seba_dos13• 5 years ago

It's the network switch that failed there. Apple adapter works only because it has less features.

hectormalot• 5 years ago

I just went through exactly this. I’d unplug my laptop, walk downstairs and suddenly the WiFi would stop working.

To troubleshoot, I’d plug into the hub to have Ethernet connectivity, and the network would seem to work fine. Took me weeks of guessing before considering that a powered-but-disconnected USB hub could be the culprit.

Odd that this is an issue across so many vendors. Perhaps they share a chip, but still, it reads as a mistake on the Hub as well as the Switch side of things.

procarch20192• 5 years ago

USB-C hubs that I’ve used have been garbage. They’ve all crapped out after less than a year or so of use. The Dell USB-C docking stations I’ve had have been flawless. Not sure what the diff is as my Dell puck USB-C hub sucked (went through 2 of them in a year), but the docking stations are solid.

The one difference I see is that the docking stations are meant to be independently powered and I think it has its own fan. I’m sure the internals are different too, but I don’t know how. Curious if anyone knows more about the difference between the hardware/firmware on the hubs vs the stations.

cptskippy• 5 years ago

I have used two Lenovo Thunderbolt docks, a Cable Matters, and a Dell; they all had some odd behavior but the Dell was by far the best.

I think the use of Display Port a plays a big factor. The Cables Matters and Dell docks only had dual HDMI and had far fewer issues.

tgpc• 5 years ago

Belkin TB3 Pro dock customer here. Is it garbage? no. Does it like to go wrong in new and mysterious ways every couple of days? yes

seba_dos13• 5 years ago

The article is about crappy network switches though, the USB-C hub did nothing wrong :P

tapland• 5 years ago

My Dell hub keeps randomly turning off the DP monitors. Cooling it hasnt helped, and it does it with both 1 or 2 monitors connected and on various laptops and OSs

readams• 5 years ago

Note that this doesn't seem to actually be related to USB-C, but rather to a quirk of how the docking station the author was using behaved when not connected to a computer. This could just as easily happen with any dock. But maybe USB-C and Apple's dongle life has made such docks more common.

soneil• 5 years ago

I'm not sure how this relates to Apple. I have a Thinkpad, and still vastly prefer the one-cable-solves-all for monitor/power/peripherals.

oneplane4• 5 years ago

This has happened with other ethernet controllers too. It's about the fact that they can send pause frames when the host disconnects while the power is still on.

You can also do this with USB-A, get an USB-A controller that sends pause frames, plug it into a powered USB hub and connect to your computer. Unplug, watch the ethernet link still being online and suddenly the network dies. Mostly happens with Realtek USB Ethernet controllers.

This is a firmware feature so technically you can turn it off.

If you want to try it with PCI or PCIe: get a powered backplane or bifurcation board or thunderbolt-to-pci bridge. Connect/use as normal then disconnect the host while leaving power on. Same problem happens with controllers that send pause frames.

ploxiln• 5 years ago

About 10 years ago, I was working in a startup that had a hadoop cluster on some bare metal servers in a datacenter. At one point when the dataset size and sustained traffic between nodes grew to a particular level, the network throughput crashed to a trickle, until the hadoop jobs were all canceled, then network was fine a minute later.

It turned out to be pause frames! The switches were too eager to propagate them, or something? The fix was to just disable them on all NICs - TCP can handle packet loss and rate control pretty well in those conditions anyway (gigabit switched ethernet).

(I can't take any credit for the work debugging and mitigating this issue.)

Anyway, it seems this is one of those bugs that pops back up from time to time... people try to implement this bit of the spec along with the rest, but don't realize how it's really supposed to behave under stress, and it's never tested under this kind of stress...

Quollified• 5 years ago

I use a USB-C/Thunderbolt dock from CalDigit that seemingly has the same problem. Luckily they released a firmware update that supposedly fixes it several months back.

I'm guessing a lot of the manufacturers just ended up using the same chip for their docks/hubs, and some manufacturers just don't care if there was a firmware update from the chip vendor.

xg152• 5 years ago

nodes sending PAUSE message to the special multicast address 01:80:C2:00:00:01 are instructing the switch to not send them any more frames. My switch seems to honor this, but also forwards the frames to the other nodes on the network, in effect telling THEM to pause in sending frames, which would explain the observed behavior.

The what now? I've never heard of "PAUSE" frames before, but those sound like a feature just waiting to be exploited.

Do we know how common the behaviour to forward the frames is? This sounds like it could bring down any vulnerable public network of their choice by just logging on and sending that special packet...

citrin_ru2• 5 years ago

You cannot exploit this in a network which uses non-broken switches - pause frames must not be forwarded to other ports [1].

But vendors who make home switches often ignore standards.

1. https://en.wikipedia.org/wiki/Ethernet_flow_control#Pause_fr...

xg152• 5 years ago

Thanks for the info.

This malfunctioning behaviour doesn't seem that uncommon though. Here [1] is another story from 2016 with the same root cause.

I'd like to know how widespread that non-standard behaviour really is.

I could imagine that many small businesses, e.g. cafes, restaurants etc, use cheap consumer switches to manage their guest wifi. Those would seem to be vulnerable.

Another thing: Maybe this is nonsense, but couldn't you spoof PAUSE frames coming from another MAC address than your own, e.g. on wifi? If that's possible, you could disrupt service for selected other network participants even on well-functioning switches (provided you know the victim's MAC address).

[1] http://jeffq.com/blog/the-ethernet-pause-frame/

HN discussion: https://news.ycombinator.com/item?id=12339902

jzwinck• 5 years ago

Why do these hubs (or the chips they're made with) want to send PAUSE frames in the first place? What's the potential advantage vs just behaving as a switch with no other devices connected?

seba_dos13• 5 years ago

This seems like a very reasonable thing to do in order to stop wasting power.

citrin_ru2• 5 years ago

A year ago spend relatively long time troubleshooting my home network problem and the root cause was the same: 1. USB-C hub with an Ethernet port send pause frames when it is powered but a laptop is disconnected 2. TP-Link Wi-Fi router broadcasts pause frames to all ports which causes all connected devices to stop sending traffic.

TP-Link (or more specifically Atheros AR8327 used inside it) here is likely breaks a IEEE standard (may be even the most basic one - 801.2d), but when no-one send pause frames this goes unnoticed.

As workaround now don't connect power to the hub and instead connect charger to a laptop directly (if a hub is powered you can connect only one cable to s laptop to get both - power and external USB+Ethernet).

the_mitsuhiko• 5 years ago

Author here: so since I wrote this blog post I get an email once every few weeks about this. Apparently that blog post ranks quite well now for the keywords that frustrated users are typing into Google.

I have now heard that this problem happens with a lot of devices and the most common element in most of them is that people also have netgear switches on their network but not exclusively.

The most reliable workaround is to ensure that the USB-C adapter with an ethernet card in it is powered off when the laptop goes to sleep. Many do that if they are not also used for USB-C PD or whatever it's called.

USB-C Hubs and Ethernet (2020)

Related Stories