The Shingled magnetic recording NAS debacle

https://rubenerd.com/the-shingled-magnetic-recording-nas-debacle/

by Breadmaker • 6 years ago

42 24 6 years ago

sokoloff• 6 years ago

In the late 90s, Western Digital drives had a terrible reputation (well-deserved). Just 15 years later, people were very positively inclined towards WD.

I think WD bean counters are well justified in thinking “even if we get caught here, this too shall pass...”

Shortly after WD starts shipping competitive products for 5% less than anyone else, the market will adapt and most will forget the SMR debacle, just like we forgot the tragically poor quality drives of the late 90s.

AtlasBarfed• 6 years ago

Except spinning rust is staring at the prospect of flash storage eating their lunch. Flash isn't THAT far away.

If these increased areal density techs don't pan out, they are done.

zozbot2343• 6 years ago

Flash storage is not nearly as reliable as spinning rust, given comparable density. The spinning rust niche is pretty much unassailable, and now SMR lets the industry expand into tape-like use cases. What WD did was a huge screwup for the reasons OP talks about, and they had to backpedal quickly.

derekp7• 6 years ago

Also, the other two manufactures are apparently doing similar -- shipping SMR drives without any indication of what they are. At least one of them released a statement that said they weren't doing SMR on drives labeled as enterprise san or something similar, but you still take a chance if getting one of the consumer drives (and for home storage, a consumer drive may make sense, as long as it isn't SMR).

nicolaslem• 6 years ago

So it came as a surprise when sysadmins began noticing their new Western Digital Red NAS drives were dropping out of NAS RAIDs and ZFS pools owing to random write timeouts and failures.

I experienced exactly that a few days ago. I have some of those SMR WD drives in my NAS and one drive just disappeared from the ZFS pool while importing a large database. I rebooted and the drive was back. ZFS worked its magic and everything seems fine now, but it doesn't inspire confidence.

phire• 6 years ago

It's quite clear that WD never properly tested these SMR drives in actual 3-8 bay NAS workloads. Certainly not a rebuild.

Based on their initial response to this scandal, it seems at some point they defined the "small NAS" Red spec as 180 TB/year read/written per year.

Then they later certified the new SMR drives as meeting this 180 TB/year spec, probably by testing they were good for 500gb/day.

They completely ignored the fact that larger home NAS installs don't use their 180 TB/year spec at a uniform rate, but instead tear though several TB doing a rebuild then sit near idle for most of the rest of the year.

zozbot2343• 6 years ago

No NAS uses that spec at a uniform rate. Practically every NAS these days uses RAID, which implies rebuilds.

mrlonglong• 6 years ago

Goddamn MBA types making a quick buck to get their bonus. This never works out too well for the company involved. Boeing for example.

tambeb• 6 years ago

Took the words right out of my mouth.

genr8• 6 years ago

This is a warning sign that we have 0 agency over hard drives we think we own. The corporations control the actual tech being pushed, to pull a bait and switch. They also control the firmware, which is super proprietary and hidden, and can be subverted and controlled without your knowledge. The end goal is to push consumers to end up storing all their data in the cloud.

donavanm• 6 years ago

The end goal is to push consumers to end up storing all their data in the cloud.

Do you really think seagate, toshiba, and WD want you to move your data to the cloud? They dont have any particular commercial interest there. Theyre just another set of commodity input providers to the Amazon/Google/Microsoft supply chains. The same as they are to dell/synnex/foxconn/lenovo when the drive is put in a different chassis and sold to you. Wheres the commercial motivation?

Personally, if I was a commodity supplier, Id much rather sell to you with a wholesale markup than negotiate with Amazon grinding out pennies from the price.

rasz2• 6 years ago

Scam, not debacle.

LargoLasskhyfv• 6 years ago

Relevant material to think about here:

Sprites mods HDD-hack

[1] https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...

Andrew "bunnie" Huang's presentation at the 30c3 Chaos Communication Congress from 2013 about the vagaries of flash memory of all sorts...

[2] https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...

The OpenSSD Project

[3] https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...

and just from a few days ago about the remake of their website?

[4] https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...

and then remembering this:

[5] https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...

and this

[6] https://www.westerndigital.com/company/innovations/risc-v

bestham• 6 years ago

The funny thing is that SMR is perfect for storing large immutable files like photos, music, videos or game resources. What we need is a controlled way of relaying the SMR properties further up the chain. Anything that is “drive managed” is bound to fall apart sooner rather than later. I have the first Seagate 8TB SMR “Archive” HDD, and I have yet to find a way to tell the controller to “TRIM” the complete drive and start over. Without this SMR-drives essentially become WORM-drives.

The “read-modify-write” nature of copy-on-write file systems are a perfect marriage with “host managed SMR”. There was a couple of sessions on SMR during the OpenZFS summit: “Host aware SMR” [1] and “HGST SMR” [2]

[1] https://openzfs.org/w/images/2/2a/Host-Aware_SMR-Tim_Feldman... and https://youtu.be/b1yqjV8qemU [2] : https://youtu.be/a2lnMxMUxyc

projektfu• 6 years ago

The manufacturers could have ushered in a revolution in filesystems that would have taken full advantage of their new drives. If they had taken an approach like in "Venti" (Plan 9), and provided an underlying log-structured block archiving service, while placing SSDs in the front, they could have sold new appliances that never forget a change and respond immediately. Running out of archival space? Just plug in another SMR drive. A machine gets encrypted by malware? No problem, all the prior data is still on the archival drive. Need to compare this file to a version from 3 years ago? The backup was automatically saved.

A lot of the benefits are there in something like Time Machine, but this system is even more automated and doesn't require explicit backup steps every hour or day, nor does it need to remove backups as they get old.

The Shingled magnetic recording NAS debacle

Related Stories