Belkin Breaks Its Routers Worldwide, Issues Temporary Fix

http://techcrunch.com/2014/10/07/belkin-acknowledges-its-routers-cannot-access-the-internet-and-issues-workaround/

by nkrumm • 12 years ago

41 26 12 years ago

abluecloud• 12 years ago

Am I correct in thinking they use a single site hosted by them to decide if the router is connected to the internet? Did noone not think this might not be the best idea?

drewcrawford• 12 years ago

What would you suggest? Placing the question of whether or not their products continue to work in the hands of third parties?

I think there is a conversation to be had here about the proper way to tie add-on services (which are inevitable) with one-time-purchase products. Consumers want some indication of whether they can reach the Internet, but this really requires some ongoing cost on the router manufacturer's part, which cannot be recouped indefinitely from a one-time hardware sale.

we face a similar crisis in mobile apps. Apps increasingly require backend services to provide core functionality, which just isn't sustainable indefinitely with a fixed-price model. Inevitably when these companies are acquired, the backend shuts down, and people lose the product that they "paid" for.

I think in the long-term, we are going to have to move to a subscription-based model where people are paying regularly for things they used to buy one-off. But in the short-term, you can have an endless stream of VC-backed companies who can afford to take on that cost for short periods. So anyone who actually broke rank would get beaten back pretty hard by the market.

mokash• 12 years ago

Mate, what are you even banging on about?

blueskin_• 12 years ago

I think in the long-term, we are going to have to move to a subscription-based model where people are paying regularly for things they used to buy one-off.

i.e. renting software.

People don't want to rent software. Look at the (un)popularity of Adobe CC etc. to show that.

This is why online activation/backend is just a bad idea in general.

unwind• 12 years ago

You seem to be identifying what they did in your first line, then go off on some kind of wild tangent in the rest of the post.

As I read it, there is no case of any "service" or "subscription" happening here.

It's just a design that relies on a single point of failure to determine whether or not a product can perform its designed functionality, which might be a bit ... sloppy from an engineering perspective.

In general, answering "am I connected to the Internet?" without relying on things outside your control is hard, probably because the Internet is all about things outside your direct control. :)

amouat• 12 years ago

My (rather old) router pings yahoo.com as a default test. Yahoo definitely didn't make it. However, the URL can be changed and the router doesn't stop working if yahoo can't be reached.

lordbusiness• 12 years ago

I would suggest placing the question of whether or not my internet connection works in the hands of my ISP. I had a similar situation when I used a Linksys router; if Linksys had any problems at all with their cloud overhead infrastructure, my internet connection died.

All I need for my internet connection to work is my ISP's infrastructure; I want my router to work without phoning home to some bloatware belonging to its manufacturer.

Apologies; please don't down vote me for this opinion.

UnoriginalGuy• 12 years ago

That isn't really the key question, the key question is: Why when that check failed were the routers "unable to connect to the internet" inspite of actually being connected to the internet.

I'm guessing the software on the routers is designed to constantly release/renew the WAN side when connectivity isn't established. The problem with that design is that ANY disruption will cause a complete internet loss (e.g. DNS timeout, site goes down, route to host, etc).

Microsoft has a similar kind of "am I online check" however the difference is when the check fails all Windows does is display a different icon and suggest (via a graphic) that you're not on the internet.

So that to me is the real Belkin blunder, not the check, or even that it used Belkins' site. The blunder is that they had the router continuously renew the WAN side until that single check worked.

bitJericho3• 12 years ago

I'm not really sure why they needed to perform this check at all? DNS will work properly whether you have a connection to the internet or not (it will just say it failed to lookup the address you requested)

personZ• 12 years ago

The iOS 6 update broke wifi for many people for the same reason -- it would hit a certain Apple page to determine if there was through connectivity, and if that test failed it would decide that the wifi connection needed authentication. Apple fixed it by putting up the page it was expecting.

http://www.iphonehacks.com/2012/09/fix-wi-fi-connectivity-is...

It seems to be a recurring tactic.

zyxley• 12 years ago

As of late Apple seems to do better at this by using a variety of different domain names (apparently pulled from their secondary 'we don't want somebody else using our product names' set, since they include ones like 'ibooks.info'), which the phone seems to rotate through so it's not relying on a single DNS result or server.

forgottenpass• 12 years ago

Did noone not think this might not be the best idea?

Unnecessary network dependencies are all the rage in software these days.

al2o3cr• 12 years ago

My guess? Belkin was routing DNS queries from these boxes through their own infrastructure. For what purpose, who knows.

Evidence is the "workaround" posted on their status page:

"We have identified a workaround that will enable some users to get back online. The workaround requires that you set a static DNS address on the device trying to access the internet."

itazula• 12 years ago

Yep, and those static IPs are for the good ol' Google public DNS servers, 8.8.8.8 and 8.8.4.4.

mzs• 12 years ago

Nah, this is the best explanation from a reddit thread, looks like it was pinging heartbeat.belkin.com to verify network connectivity after getting DNS servers from DHCP:

http://www.reddit.com/r/technology/comments/2ik43h/belkin_fi...

10:40 AM <Duiwel> Just got an e-mail from someone on our Wireless ISP mailing list: We found the routers are sending icmp to heartbeat.belkin.com. Even though we could get a response here at our NOC, the belkins are not receiving it. We added the ip for heartbeat.belkin.com as a loopback address on a router on our network and it becomes

10:40 AM <Duiwel> reachable to all the belkins on our network. Lo, and Behold! They all work again.

10:40 AM <Duiwel> May have to try that

roywiggins• 12 years ago

That's a bit freaky. I more-or-less expect my ISP to see all my DNS queries, but not my router manufacturer.

nandhp• 12 years ago

My theory has been that if "the Internet is down" (ping failure to heartbeat.belkin.com) the router does DNS hijacking to show the user a diagnostic page. However, I haven't found a description of what the problem looks like from the Belkin-user's end, so I have no evidence to back that up.

bitJericho3• 12 years ago

More reliable news source: http://arstechnica.com/information-technology/2014/10/borked...

drcross• 12 years ago

To break it down- the routers use a Belkin IP address as a DNS relay, when it couldn't reach that address the router responded with the error. So the problem was that their internet facing DNS server went down. It's pretty poor that they dont have backup DNS servers but it's not the end of the world it's not as if their backdooring fubarred firmware onto the routers.

halviti• 12 years ago

Not according to the article. Belkin claims that it was a problem with the server they used to allow the router to check for internet connectivity.

If it comes out that Belkin is actually routing their users through their own DNS, I would hope it would be a huge privacy scandal.

There is no reason for them to run a DNS infrastructure for their clients, and doing so provides them no direct benefit, has no clear applications for building routers at all, costs money to maintain, and is a huge point of failure.

I can't imagine them doing such a thing for a few advertising dollars.. the only reason a router company would do that is if they were paid by the government or something. This way Belkin could reroute your internet traffic through whoever's servers whenever they wanted, and you would likely never know.

Anyway, that sounds a bit far-fetched, so I'm going to hope that Belkin is telling the truth.

bitJericho3• 12 years ago

They are because the ISP is supposed to provide DNS. That's what you're paying the ISP for. If the ISP paid belkin to run the DNS servers then yeah, that's no big thing, just a service outage.

makmanalp• 12 years ago

Want to hear a fun story?

You remember dyndns.com ? Well, it used to be a small app run literally from a dorm room at WPI. Well, after a while, it became one of the largest dynamic DNS providers in the world, and router manufacturers around the world started supporting it in the UI. Dyndns used to provide an address (an olden-days plaintext web API of sorts) to check the external IP of a router. Eventually, manufacturers realized that even if they didn't have a deal with dyndns or even formally support them, they can use their IP address to check the router WAN IP. Major manufacturers like linksys and d-link (I think, don't take my word for it) jumped on the bandwagon and everyone started pinging the crap out of that page and if they took it down, these routers would break in interesting ways. So they built it up and kept it running.

That API runs to this day, at checkip.dyndns.org.

edit: Today that company is dyn.com, bootstrapped pretty much all the way, and provides DNS to twitter, etsy, fastly, the guardian etc.

arbernat• 12 years ago

It's very easy to cause a ton of problems with commodity routers. The University of Wisconsin - Madison CS department got DOSed back in 2003 by Netgear routers using the CS NTP server because it was hard-coded into the router firmware. Full story is linked below:

http://pages.cs.wisc.edu/~plonka/netgear-sntp/

cesarb• 12 years ago

This event reminds me of an old definition for "distributed system":

"A distributed system is one in which the failure of a computer you didn't even know existed can render your own computer unusable." (source: http://research.microsoft.com/en-US/um/people/Lamport/pubs/d...)

In this case, the failure of a system most people didn't even know existed (Belkin's heartbeat server) rendered their router unusable.

Belkin Breaks Its Routers Worldwide, Issues Temporary Fix

Related Stories