Tape still beats SSDs and hard drives when it comes to price per byte

https://www.techradar.com/news/theres-one-crucial-way-tape-still-trounces-ssds-and-hard-drives-when-it-comes-to-storage

by jaytaylor • 7 years ago

230 277 7 years ago

techradar.com

basicplus23• 7 years ago

"As for transfer speeds, they can reach 300MBps"

I am getting over 500MBps on my setup

jaytaylor2• 7 years ago

Impressive.

Will you share more information about the specific details (i.e. hardware and configuration) for the set up you're running with?

merb2• 7 years ago

well probably with compression

basicplus23• 7 years ago

Ok now i am embarressed.. I was out by factor of 10...

i am getting 50Mbps.. copying NAS to tape

For what its worth..

Network Router: TG789vas v2

NAS: Synology DS1815+

Switch: (for Server, Tape Drive, Tape Library & Their Associated UPS ) Airpho Gigabit AR-GS108

Server: X3650 M3 Running Windows 2012 R2

Tape Library: IBM TS3100 3573 L2U

Tape Drive: IBM LTO6 8Gib Fibre Channel 35P1624 Feature Code 8348

Tape Drive Fibre Channel Card: Avago AFBR-57F5PZ

Server Fibre Card: Emulex LPE12002

Using NO compression

I understand the new IBM LTO 8 Drives can get up around 750 to 800Mbps

snoman• 7 years ago

Mind sharing the details of your setup? Interested in doing something similar.

adrian_b22• 7 years ago

I have been using since 2016 the following setup:

1. QUANTUM LTO-7 Tape Drive, Half Height, Tabletop 2. Delock Cable Mini SAS HD SFF-8644 > Mini SAS SFF-8088 1 m 3. LSI LSI00343 SAS 9300-8E Host Bus Adapter (8-Port, PCI-e 3.0 8x, SAS 3.0)

Also, there is a couple of useful tape accessories:

4. Cleaning Cartridge (I use one from Quantum) 5. Cases for storing LTO Ultrium cartridges (I use some from Turtle Case, for 20 cartridges each)

basicplus23• 7 years ago

Ok now i am embarressed.. I was out by factor of 10...

i am getting 50Mbps.. copying NAS to tape

For what its worth..

Network Router: TG789vas v2

NAS: Synology DS1815+

Switch: (for Server, Tape Drive, Tape Library & Their Associated UPS ) Airpho Gigabit AR-GS108

Server: X3650 M3 Running Windows 2012 R2

Tape Library: IBM TS3100 3573 L2U

Tape Drive: IBM LTO6 8Gib Fibre Channel 35P1624 Feature Code 8348

Tape Drive Fibre Channel Card: Avago AFBR-57F5PZ

Server Fibre Card: Emulex LPE12002

Using NO compression

I understand the new IBM LTO 8 Drives can get up around 750 to 800Mbps

jolmg9• 7 years ago

Even if LTO-8 tapes are now in stock, you can buy cheap LTO-7,

I wonder if this was a typo. Did they mean that LTO-8 tapes are out of stock? Otherwise, the sentence structure "even if ... are now in stock, you can buy ..." is kind of weird.

reformat them to M8 and get 9TB of native storage (22.5TB compressed).

What does this mean? Doesn't how much it compresses depend on the data not the storage device?

bengerbil• 7 years ago

There was a dispute involving LTO-8 that just recently was resolved, so LTO-8 tape should now be available in the US. I wasn't following it closely, so I can't give any real insight, but it explains the wording.

mycall• 7 years ago

3TB for $153. Not impressed.

https://www.google.com/shopping/product/10502190238469223139

wtallis• 7 years ago

Until very recently, LTO-8 tapes were not available for purchase due to ongoing patent litigation. LTO-8 drives were readily available, but you couldn't buy the tapes to make the most of those drives.

LTO-8 drives can also use blank LTO-7 media to get density halfway between LTO-7 and LTO-8; this is referred to as LTO-7 Type M, or Type M-8. This applies to the raw storage capacity, not just the compressed capacity (which is conventionally referred to with the assumption of a 2.5x compression ratio, even though not all real-world data is compressible).

jaytaylor2• 7 years ago

I'd be curious to hear from any folks who actually run tape drive setups for personal use.

Where did you get the gear?

How cost-effective is the overall setup compared to spinning rust?

How is driver support in popular operating systems?

Is there more you'd recommend taking into consideration or learning before jumping in? Especially beyond the obvious realities of extremely high latency, of course ;)

netrap• 7 years ago

eBay. 1-1 HDD backups is probably close in price even in the used tape market. I use LTO-4 with FC. Works fine in Windows with Retrospect. I'd expect it to work in Linux as well. The hard part is backup software. I guess with LTO-5+ you can just directly access the tapes with LTFS, but I haven't tried that yet.

mc32• 7 years ago

Anything which supports ndmp should be good. Both bacula and Amanda among other SW support NDMP. I mean of course enterprise solutions support it too.

klodolph6• 7 years ago

Where did you get the gear?

I never ended up using it personally, but when I was considering it, used gear on eBay was top of the list.

How cost-effective is the overall setup compared to spinning rust?

For personal use, it is not even remotely cost-effective unless you are a data hoarder. The cost of the drive is simply too high. Also keep in mind that the drive can become damaged or need servicing.

It only makes sense if you are dealing with e.g. large quantities of video. The article cites a minimum 140TB where tape becomes cheaper but this involves some dubious decision-making.

How is driver support in popular operating systems?

I have only ever used tape with Linux + custom software so I can’t speak of software support in general.

Is there more you'd recommend taking into consideration or learning before jumping in? Especially beyond the obvious realities of extremely high latency, of course ;)

The obvious reality to me here is that it just doesn’t make sense unless you’re in spitting distance of a petabyte. The only thing I can imagine someone doing personally is video, and it would have to be a lot of video like some kind of long-running documentary project that you are producing in your spare time.

The other factor is to be more paranoid about mixing tape generations than the vendors suggest. Ignore the specs and stick to same-generation media and drives.

Efficiently writing to a tape means filling e.g. a 20 MB/s write channel for LTO-6, or 750 MB/s for LTO-8. Can you sustain 750 MB/s? It’s not so easy in practice. You throw away tape capacity if you get buffer underruns.

jolmg9• 7 years ago

Can you sustain 750 MB/s? It’s not so easy in practice. You throw away tape capacity if you get buffer underruns.

The drive doesn't pause while the buffer fills again? Considering the price of the drives, I find it very surprising that they don't do that.

Dylan168076• 7 years ago

Can you sustain 750 MB/s?

Do 10% of a tape at a time, buffered on a $200-$300 NVMe SSD?

ijiiijji12• 7 years ago

I have, back in the day.. both floppy-controller and IDE based... with a whopping range of storage from 220 MiB to 4 GiB.

Since I don't have enough karma to write another comment anonymously:

Restoring from backups is a last resort for very important files deleted more than a few days ago (beyond what hourly/daily snapshots contain) or for business continuity (BCP) / disaster recovery (DR) of an entire system that cannot be recovered via automated configuration management rebuild. Snapshots to nearline storage reduce RTO for most common use-cases, and most users and systems don't generate much churn of data in a day. For recovery of important files, sending a courier out to retrieve a backup tape set from the vaulting provider implies the RTO would accept in a matter of several hours. An RTO of under an hour would imply backups would need to be stored on nearline media (commodity spinning rust) or at a backup cloud vendor like AWS Glacier or Tarsnap, which is more expensive than tape.

The deciding factor of how much to rationally spend on DR/BCP solutions follows from a Business Impact Analysis (BIA) to quantify the loss of a service over a certain time, which drives RTO and RPO goals. (A common factoid of DR/BCP is "about half of businesses who lose their data go out of business within six months.")

RTO - Recovery Time Objective - desire/need to recover within a certain time window

RPO - Recovery Point Objective - how close to the original is the recovery

TwoNineFive4• 7 years ago

Where did you get the gear?

I have an old Quantum DLT S4 drive and something like 18 tapes. Each tape is 800GB uncompressed and I get about 1.8x compression on average. It's an external unit connected via SCSI ultra 320. The ultra 320 SCSI card is something I picked up from ebay for like $20. The drive and tapes I got from a former consulting client who was going to trash it.

It was the last of the DLT line and is similar to LTO4. Get an LTO5 if possible; it has some cool features like partitions.

I only store my most critical stuff on tapes. Each weekend I do a tape swap and take the old tape to work on Monday and bring the oldest one back. I keep the tapes at work in case my house burns down or something (I lived through two house fires as a kid so I'm super neurotic about it, don't ask). The tapes are encrypted so nobody can recover from them if they get one. If my house did burn down I'd have to somehow procure another tape drive, but that would be the least of my problems.

I just use a custom tar script to do it. I tried both Bacula and Amanda in the past but they were too complicated or had other issues. My own tar script did what I wanted.

How cost-effective is the overall setup compared to spinning rust?

Since I got most of it for free and I already had experience with tape, it was pretty good. I've had it since 2008 so it's over ten years old and still going. I also have some UBS3 backup disks that I use for daily backups. In all this time, only one tape has died and I do a full validation of the previous backups every time I bring a tape back and re-use it.

For new installs, go with some USB3 disks from Seagate or WD -- both are great, I have both. Avoid Toshiba due to some weaknesses in handling bumps and vibration during runtime due to firmware (Toshiba internal drives are great though) -- Don't ask me how I know this. Seal them into some kind of fire-proof/flood-proof safe and store somewhere that a fire/flood won't destroy it.

How is driver support in popular operating systems?

I'm all linux. It works. All that matters is the SCSI card being supported. Make sure NOT to get a RAID card, regular SCSI only. The RAID cards often won't work with tape drives because the firmware is made for RAID drives, not tape.

Is there more you'd recommend taking into consideration or learning before jumping in? Especially beyond the obvious realities of extremely high latency, of course ;)

So the only real advantage for me using tapes is that I got most of it for free, the tapes are more portable and lighter than a USB hard drive for carrying around. Almost everything else is a negative compared to USB hard drives.

I use some UBS3 hard drives for daily backups via an rsync --link-dest script that de-duplicates everything and that works great. For $100 you can get a 4-6GB USB hard drive from WD or Seagate and for my important stuff, that works fine.

I'm about to fill up my older 5GB drive -- it's down to about 200GB free space left. I already have a new 6TB drive I brought recently and when it gets down a little lower, I'll just replace it, do full backups, and put the old drive into storage for permanent archival.

Data backups is a full occupational specialty, like being a DBA, network engineer, security, or web monkey. I did TSM for awhile. There's a ton of issues in the hardware, strategy, security, performance, and more.

Do yourself a favor and just set up something simple that JUST WORKS and don't fuck with it too much.

Do a practice recovery once and that'll teach you how good your setup is and improve.

craftinator• 7 years ago

I am incredibly impressed with your level of s/paranoia/detail. What do you do for a living that requires that much rigorous static data retention.

sshagent2• 7 years ago

As a backup specialist that last line of yours is everyone should pay attention to. It sets expectations on recovery time and proves it work.

We ( work ) have a massive pile of DLT, mainly S4's...these were written 2005 to 2009 ish and they still restore just great. I mean, i wish they wouldn't as i'd love to be shot of them but they are too reliable.

cf141q53253• 7 years ago

May I ask what kind of encryption you use with the tapes?

oconnore• 7 years ago

For personal archival I think Blu-ray is still best. I think of it as a supplement to S3 or Dropbox. Even if you end up re-writing discs it is very cheap per GB, and 25 or 50 GB discs are comfortable for “normal” personal sizes (my entire file set that I care to keep around is around 100 GB — mostly because I’m not careful about deleting old node_modules from ancient projects I’ve done)

jbverschoor2• 7 years ago

My photos/videos alone are over 300gb and growing.

node_modules is shit anyway, and shouldn't be duplicated for every single project.

scarejunba3• 7 years ago

What's Blu-ray durability like with commodity drives?

Erlich_Bachman5• 7 years ago

Note: there are 100GB bluray discs now that can be recorded using consumer burners. There are also M-Disc bluray discs which are specified to last about 1000 years (with some science and testing to back it up, even though no one tested it for 1000 real years obviously), because it uses a more hard "rock" layer to store the data instead of basically a color dye which is used on regular burnable discs (which last for 10 years according to some sources).

ramraj072• 7 years ago

I setup a tape drive for my lab backup and in paper it was straightforward- we got a tape drive from Newegg for a few thousand bucks, put it in a machine, realized we needed some pci card, got it, realized we needed some cable, got it, realized we needed some custom drivers for some reason, got it, and then finally it was working. Started getting some tapes and backing up. Overall would not recommend unless you have hundreds of terabytes that you wanna backup yourself

mmaunder3• 7 years ago

LTO is used in the film industry for long term archival. I own a LTO8 drive. Sourced from Maxx Digital in Orange County, CA. Talk to Matt Stone and he'll spec one for you. Tell him Mark Maunder sent you. :-)

The drives are bloody expensive. Mine was around $5k.

LTO8 tapes are still hard to come by even though the patent dispute is settled. This should change soon.

The software to archive to tape is clunky and frustrating but it works. I use something free that integrates with Finder on Mac. I'm away from my office so dont remember what it's called.

The main benefit is that the tapes last for half a century and are very cheap per TB once you have made the initial investment in the drive. Very good for long term archival of films and source footage which is what I use it for.

Keyframe• 7 years ago

Exactly what we’ve used it (when I was day-to-day in that business as well). Daily shot material stored in three copies and sent to three different physical locations and one additional copy for on-site archiving + online on hard drives.

When you do a lot of those, drive price isn’t all that expensive. My major gripe with it is that you couldn’t hook it up to a USB3. Rare ones had thunderbolt, but most were that bloody external SAS thing. Not really all that great when you don’t have those controllers around. Beats the purpose of external drives.

adrian_b22• 7 years ago

I have been using in my home for several years a Quantum external LTO-7 drive, which is connected via a Serial-Atached-SCSI cable, so I also have a SAS HBA card in a computer. I am living in the European Union and I have bought the required hardware from an on-line shop from Germany, where it was cheaper. Now you can also buy it easily e.g. from Amazon. The cost was around 3000 EUR.

The tape cost is about 10 EUR per terabyte. Previously I was using HDDs for archiving. I have not followed closely the evolution of the HDD price, but looking now on Amazon I see about 30 EUR per terabyte.

I have a data archive of about 120 TB, but I am paranoid because of past data losses due to defective HDDs/CDs, so I store 3 copies of each tape (in different places).

That means that by using tape instead of HDDs, I have already saved 20 * 360 - 3000 = 4200 EUR.

Someone who would store just 2 copies of each tape (the minimum rational value), would need to have a data archive of at least 75 TB to recover the initial cost.

Nevertheless saving money is not everything, you also gain peace-of-mind because the chances of the tape becoming defective just from storing are negligible compared to a HDD, which if not used for a very long time might never start spinning again.

With tapes you must transfer the data only when you can no longer buy drives that would read them. With HDDs you should better replace them before their warranty expires, as I have seen too many that died immediately after that.

The tapes work OK both in Linux and in FreeBSD, but the FreeBSD utilities are more convenient, so I have connected the tape drive to a FreeBSD server and I transfer the data to be archived through 10 Gb/s Ethernet, which is faster than the 6 Gb/s SAS, so it is not a bottleneck. The writing speed of LTO-7 is 3 Gb/s, so that is the bottleneck anyway, not the interfaces.

That speed is much faster than of any HDD of up to 8 TB that I have ever used, but I have not checked the latest HDD models to see if they have improved their speed to levels comparable to tapes.

Using tape is very simple and retrieving any file (I have a tape index on my SSDs) takes at most 3 minutes, including the time to manually start the drive and inserting a tape in the drive. The maximum seek time is about 2 minutes. When I had a HDD archive, there was still the same time for manually selecting a HDD and inserting it in the HDD rack. For retrieving very large files, the tape can even be faster, due to the faster sequential speed.

The conclusion is that if you have data of at least several tens of terabytes and/or if you value that data enough to want to avoid any chance of losing it, than magnetic tape remains the best solution for now.

Annatar8• 7 years ago

Got it on ebay. There are great deals to be had if one knows the technology, partly because there are very few of us left now who know it, so extremely expensive equipment goes for pennies on the dollar.

I use self-system engineered (read: self OS packaged) Bacula software for backups with a custom developed shell program which interfaces with Oracle RMAN to do backups.

I use Solaris 10 on sparc, which also goes for pennies on the dollar nowadays since there are few of us left who understand how to run it and even fewer who understand how to squeeze the most out of that hardware + software combination.

The tape library shows up as character and block files under /dev/rmt/ and the robotic arm with the barcode scanner under /dev/changer/, nothing special or fancy about it. There is another piece of freeware I compiled and packaged (whose name I forgot) which controls the robot arm and the scanner, also by issuing standard SCSI commands to /dev/changer/ files, nothing fancy or special about it. The tape library has enough capacity for full backups with three month retention policy, so the only thing I have to do is replace a worn out tape cartridge roughly once in three years. Backups are daily. The entire solution ran me around $800 (server + differential SCSI controller + tape library) and the tapes ran me $500, over the span of 12 years.

TheCondor• 7 years ago

LTO6 here, I bought it at Newegg. It’s pricy but having a 2.5TB and fairly durable cartridge is nice for my needs. I just dropped one earlier this week, no big deal though. I use LTFS on it, it has latency but it’s faster than you expect and once you’re streaming on or off of it, it’s downright quick. Historically, I burned optical media for this but the size of tape makes it much easier. It’s on a SAS card in a linux server, I literally plugged it in and it worked with open SUSE. I forget if the tape tools were already there or not but the old school Unix stuff just worked. I installed the ltfs tools and it all worked without any tricks. It’s a quantum drive. I did experiment with tar initially but ltfs makes it radically more usable, radically.

Pre-tape, I used disks, rotated them and kept them in a safe. Burned optical discs of my most important stuff. I had two experiences that caused me to change: I had a failure and went to do a recovery and in either haste or anxiety (I put this on me, I should have paused and took more time but I was rushing) I knocked the most recent drive off my desk and effectively killed it. The other was a bit more subtle (oddly, I have only had 3 backup recovery situations and I messed up 2 of them) but my “backup” was rsync to disk on cron and a backup started during restore and began to “remove” my good backup because the master didn’t have that restored yet. There are a lot of lessons in there, most significantly, “backup” is a different thing than live access to data, you want it versioned and somewhat fixed. Rsync isnt it. You don’t want to accidentally delete your backup. Tape switches your mindset on that stuff. I’m not going to pretend it’s cost effective for my needs (although I have a lot of hd+ video of my kids) but it’s durable, it’s purpose built and cold storage can’t really be hacked. There’s no monthly fees, no risk of a cloud provider sunsetting my tier of service, and 25T of tape is like $250. Drive MTBF numbers are big enough to be abstract and yet you will likely encounter it at some point, tape has these practical numbers that are ultra conservative and come from decades of experience; you sort of keep it in mind when you’ve used a tape a dozen times or so. That being said, when the drive flakes, I’m in for an expensive upgrade/replacement and when I do upgrade, I should copy my tapes to new ones with the new format which will take time and money. With SMR, it’s almost like discs are taking on tape like qualities and it’s not hard to find stories about people getting crap performance on their new disks. It’s not sexy but I feel very confident that I’ll be able to get my data if I need it. It’s really easy to store a tape in your safe deposit box or at your parents house too, they won’t accidentally kill it by knocking it off a shelf.

tomatocracy• 7 years ago

I also run my own tape setup for personal backups. On where to source etc - I’d agree with others that eBay is a good option for both drives and media (but as always check what exactly you’re buying and make sure you’re comfortable with the seller). You’ll almost certainly need a SCSI or FC interface too so don’t forget that. I’d recommend going with external standalone drives rather than either internal or sled mounted for this type of setup unless you’ve got a good reason to do otherwise as that means you only need to power the drive when you’re using it which can cut down on wear etc.

I’ve only ever read bad things about mixing generations despite the fact that the drives are capable of this (eg LTO5 tape in LTO6/7 drives) though I’ve not done this personally - but I wouldn’t recommend doing so based on this. Don’t be tempted to buy really old generations as if you ever need to restore in a disaster where you only have the tapes, you then have the nightmare of sourcing a new old drive too which by that point might be a lot harder.

You’ll want to think carefully about software setup. You’re probably going to wind up using some form of custom scripts unless you can put in the time to set up bacula properly and are sure it works for you (bare metal restore can be a tricky aspect of any system to verify it works properly). If you script your own then make sure you also understand the drawbacks of tar. tar as a tool is highly compatible which is good but is poor at recording incremental backups (there are some custom extensions which gnu tar has but this reduces compatibility) if you want to go with a more complex schedule than full backup every time. Tar also makes it difficult to seek to just the part of the archive that you need to get a single file. Again there are gnu extensions for this but this isn’t the best solution. Finally, tar then gzip means if your gzip stream gets corrupted then you might lose quite a bit of the tar archive so it’s not very resilient. You could rely on the drive compression but you might also consider other solutions, especially ones which also include checksumming. Consider either scripting around these or better what other tools could take its place (eg I have used dar in some of my setup as it handles this better but then you need to think about keeping a working binary around which can unpack your archives - I put a static compiled binary in my archive sets just in case).

Finally if you go down the tape route then consider strongly spooling your stream which will be written out to tape onto disk first, to give yourself maximum chance of getting the more consistent throughput onto tape which has benefits in terms of tape reliability (and may indeed wind up being quicker overall). That also allows you to easily write multiple copies of the same tape if you intend to send one or more offsite and keep one locally.

Where this setup works well for me is that when you start wanting many copies of your backups (offsite, multiple point in time full backups etc) then the cost effectiveness becomes a lot better than the equivalent with disk. I also trust tape to survive longer term on a shelf than disk.

robjan• 7 years ago

Another crucial importance is bytes per square centimetre. When archiving something, there's no point in archiving hundreds of storage controllers with it especially if you need to store them in a safe (which you probably should). MicroSD cards may beat tape but they are a lot slower and harder to label.

koolba• 7 years ago

Which one wins on long term storage durability, shock proof, and water damage?

blotter_paper• 7 years ago

https://en.wikipedia.org/wiki/M-DISC

bt8482• 7 years ago

LTO doesn't suffer from those things. Shock can destroy the case and dislodge the leader pin of an LTO cartridge but AFAIK it can't damage the data on the tape. Water is bad for the transport but as long as you dry your tapes out they should still work. The operating specs for LTO are crazy, you can read and write them under all practical atmospheric conditions.

rodgerd2• 7 years ago

LTO turns out to be garbage on the key measure of "can I read my backups a little while down the road", because you'd be surprised how quickly a new LTO drive can't read old tapes.

It's unpleasant having to buy old drives to keep with your tape archives because you won't otherwise be able to read them any more.

sliken11• 7 years ago

I had a room full of servers, many with 16-36 disks per server and had an AC failure. Things turned off as planned, but it hit 115F for a bit. None of the disks died, majority of the tapes did.

philjohn• 7 years ago

Tape vs SD cards? Tape.

NAND flash can leak charge over time if not powered on which makes it unsuitable as an archival medium. Depending on P/E cycles each cell has undergone and the drive type (SLC/MLC/TLC/QLC) it can be from 3 months to 10 years.

techntoke3• 7 years ago

Price per byte maybe, but in value you are sacrificing a lot. I prefer continuous backups and versioned storage.

viraptor2• 7 years ago

You're not sacrificing anything. Or specifically having a long term, offline tape storage does not prevent you from having short term, online, versioned file/block storage. Different use cases.

techntoke3• 7 years ago

Well, my online versioned storage via Google is unlimited for $12/mo which syncs my files locally and then I have another backup provider that does the same thing via CrashPlan Small Business which is also unlimited for $10/mo. So that is $22/mo for unlimited continuous backup and versioning. I don''t see how tapes would offer much value, and storage location of those tapes especially for a home use case is likely going to be a single point of failure.

mrfusion• 7 years ago

What about tape gives it superior compression? Why can’t that same compression be applied to any storage?

jarfil• 7 years ago

Nothing. They estimate an average compression based on an average dataset with uncompressed text data. That wouldn't work for things like already compressed video, audio, encrypted data, and so on.

cl0ckt0wer• 7 years ago

works great on giant uncompressed database backups though

userbinator4• 7 years ago

I really wish they wouldn't add stuff like compression and so forth, because I feel like it's an "abstraction layer violation" and it's misleading in a way that the majority of other storage media isn't; for example, a 1TB hard drive is going to store (at least) 10^12 bytes regardless of what those bytes contain. A 60 minute audio tape will always record 60 minutes of pure silence, white noise, or anything in between. Yet somehow data tape is marketed with a theoretical compressed capacity.

jolmg9• 7 years ago

The compression ratio seems to actually be much worse than gzip, but it's also faster.

makapuf2• 7 years ago

on a core i5 you can use lz4 and compress LZ4 for a ratio around 2 at 370 MB/s (ans decompress it faster) - for one core.

userbinator4• 7 years ago

It's not about compression, but total storage capacity.

Imagine a disk. It has two sides and a certain surface area, as well as a certain density (bits per square in/cm). A standard 3.5" HDD platter has a bit more than 4200mm^2 of surface area (having been unable to quickly find the dimensions of an HDD platter, I grabbed the dimensions of a floppy disk from the standard) per side, or a little less than 10000mm^2 per platter; I'm being deliberately rough with the numbers since all we need is an order of magnitude.

Imagine a tape. It's long and thin, and can be rolled up. An LTO-8 is 960m long and 12.65mm wide, for a total surface area of 12,144,000mm^2. Tape has a lower recording density than HDDs (LTO-8: ~20Kbit/mm * 6656 tracks, or ~128MB/mm^2 compared to ~1Tbit/in^2 = ~185MB/mm^2 for a HDD), but more than makes up for it with its length.

caf• 7 years ago

After multiplying the single track linear density by the number of tracks, you also need to divide by the tape width to convert to a recording area density number. So you end up with about 10.9Mbit/mm².

sliken11• 7 years ago

That's all well and good, but tapes are smaller than disks. LTO-8 is 12TB and the 14TB disks have been out for a good while.

Also with inefficient tapes you end up spending many more tapes for inefficient incrementals, incredibly long seeks, and lack of efficient deduplication.

Then there's other nasty surprises like a relatively narrow temperature range for tapes and an improper alignment of the head can result in tapes you can read and write... but when the drive dies you can't read the tapes anywhere else.

Tapes still make sense above a certainly scale, but I think it's around 10PB before the real world costs of tapes, drives, robot, support agreements, extra capacity for inefficient incrementals/dedupe, etc.

The big win with tapes is it's much more feasible to rotate tapes offsite.

jonah-archive• 7 years ago

If folks are curious about the benefits/drawbacks/etc of the medium, I would highly recommend checking out the media presentations given at the Library of Congress Designing Storage Architectures meeting (most slides are online -- scroll down to the "Designing Storage Architectures Meetings" section and click through to each year's list of presentations): http://www.digitalpreservation.gov/meetings/index.html

The 2019 meeting just happened and had a lot of good info about current use of tape, slides should be online in another couple weeks. (I presented there this year and work at the Internet Archive, though we do not currently use tape in our storage infrastructure.)

brador• 7 years ago

slides should be online in another couple weeks.

Why does it take so long?

hamandcheese• 7 years ago

They have to be loaded from tape.

pletnes• 7 years ago

The tape has to rewind.

jhoechtl• 7 years ago

People have something better to do instead providing information to you for free?

ghaff• 7 years ago

Because someone probably has to take time out of their day job to do so. Also, if it’s like any other conference I have ever attended, half the presenters didn’t send in slides and have to be hounded to do so—and many still won’t get around to it.

dvfjsdhgfv• 7 years ago

For the impatient, could you summarize it for the tape?

zaarn4• 7 years ago

Basically, tape has a very very high latency (on the order of minutes to hours) for random access and it has a very low number of writes (usually a few dozen) before you need to copy it to a new medium.

The drives are also quite expensive and if you use them for backups, you need to keep one ready in case the other borks it but you need the tape's data.

Lastly you need specialized tools for access since most average tools will mishandle the tape.

fiedzia• 7 years ago

If you are impatient, tapes are not for you.

vadansky• 7 years ago

Is Tape worth it for a movie collection? I don’t think I will see any benefits of the compression and the prices seem comparable to just buying a 8TB HD

klodolph6• 7 years ago

Is Tape worth it for a movie collection?

I doubt your movie collection is large enough that tape would be cheaper. A single tape has about ~50x the capacity of a Blu-ray Disc, but you need to buy a lot of tapes before the whole setup becomes cheaper than HDD.

I don’t think I will see any benefits of the compression…

Ignore the compression, it’s garbage.

…the prices seem comparable to just buying a 8TB HD…

By “comparable”… tape is clearly cheaper per byte when you compare prices, it’s just not so radically cheaper that it changes your life.

adrian_b22• 7 years ago

The most common tape is LTO-7, which is 6 TB (non-compressed). The most common Blu-ray disks store 50 GB, so a tape equals 120 discs, not 50.

I have mentioned in another message that the breaking point is around 75 TB or around 50 TB, depending on how many copies you store, i.e. around 1000 to 1500 Blu-ray movies.

I agree that many people, probably most people, do not need to store so much data to make tape storage economical, but for those who have a large enough data archive, like myself, the tape is really radically cheaper to have changed my life.

Obviously, there is a vicious circle, few people buy tape drives because they are too expensive, so that tape drives are expensive because few of them are sold.

If they would have been mass produced, tape drives should have been 5 to 10 times cheaper, and then tape archives would have made sense even for the smallest movie collections.

adrian_b22• 7 years ago

Tape (LTO-7) costs 3 times less than the cheapest HDDs, so the prices are not comparable at all.

To recover the drive + SAS HBA card costs, the movie collection must be larger than about 75 TB if you store double copies or larger than about 50 TB if you store triple copies.

If you store a single copy of any digital medium, the chances to not lose anything after many years are negligible.

adrian_b22• 7 years ago

I forgot to mention that I referred to the prices for non-compressed tape storage, i.e. counting LTO-7 tapes as 6 TB tapes. Compression does not work for movies and even for compressible data I disable the tape compression and I use better compression algorithms on the computer.

magicalhippo2• 7 years ago

Here the LTO-7 is only half as cheap as a HDD, ~110 USD for 6TB tape vs ~230 USD for 8TB HDD (cheaper per GB than 6TB models).

Cheapest new LTO-7 drive is over 3000 USD though, which means you'd need more than 300TB of data for a new LTO-7 tape solution to be cheaper here.

sliken11• 7 years ago

Beware of two things with tape. They often have a VERY low storage temperature. I lost $5k in tapes with a minor AC problem and the room only got up to 115F for a bit.

Also keep in mind that capacity and bandwidth often assume 2.5x compression of pure text, which doesn't seem like a common scenario.

The article compares SSDs to tape is silly, access times are in the minutes when a robot is involved, and many minutes of you are swapping trays. Why compare to SSDs with that can to 50k IOPs and up?

abalone• 7 years ago

Did anyone ever figure out what Amazon Glacier uses? They only publicly talk about using custom software to "optimize the sequence of inputs and outputs"...[1]

[1] https://aws.amazon.com/glacier/faqs/

cobookman• 7 years ago

Note cloud vendors get the benefit of mixing high IOPs workloads with low IOPs workloads.

Say a cloud vendor needs a bunch of HDD for a database workload with high IOPs. Using modern disks (4+ TiB) this leaves a lot of stranded disk storage that can only be used for low IOPs workloads.

Archive storage uniquely meets the criteria of lots of storage, yet low IOPs.

Disclaimer work for Google but have no knowledge of how AWS glacier works.

ggm• 7 years ago

I could believe that we have a lot of unused space which is amenable to a different overlay use, but the potential for mis-design is huge here. If I pay high IOPs cost and you have to replace my high IOPS data backend on my paypoint, migrating the low paypoint secondary customer is non-trivial if not designed in day one (I am sure you did: cheap players might not understand the risk exposure here)

The telco equivalent was ISDN, where a single linecard with two sub-rate ports might be sold to a bank, and a domestic user. The bank could (and would) demand card reset 24/7 to restore service to their ATM, the domestic user simply lost carrier, and game state.

colanderman• 7 years ago

Rumor has it Blu-Ray (specifically BDXL): https://storagemojo.com/2014/04/25/amazons-glacier-secret-bd...

Though a comment on that page mentions this HN comment claiming custom low-speed HDDs: https://news.ycombinator.com/item?id=4416065

sshagent2• 7 years ago

I certainly recall a Facebook video showcasing their bluray archival system. I was seriously impressed by the whole setup. And here we are archiving to LTO like primitives :)

milesward• 7 years ago

:) I know :)

etaioinshrdlu• 7 years ago

Rumor (from this site) consensus seems to be custom low RPM hard drives optimized for long life.

marrickvillain• 7 years ago

Disappointing article. Sure $/GB is a factor. But what about RPO and RTO? Or $/IOPS? When those factors are taken into consideration, disk often wins. Which is why the backup to disk market has gone bonkers in recent years and tape hasn't.

ijiiijji12• 7 years ago

I worked at IT for a large university. We had an HP/Compaq SSL2020 with dual AIT drives made by Sony. The issue was never determined (perhaps micro debris in the air), but the drives has to be replaced nine (9) times due to failure. Furthermore, backup and verification is a tedious process, and combined with local and offsite vaulting and tape management, having proper backups is a royal PITA but there's no substitute. Replication isn't a backup because it propagates errors and you can't do disaster recovery without proper software, offsite vaulting and practice.

Backups aren't valid until they're tested.

userbinator4• 7 years ago

IMO one of the worst parts about LTO is that it's another one of those formats with "open" in its name, but you can't just download a PDF of the standard or even buy it from something like ISO/IEC. There's some convoluted licensing process involved. Only the first generation was actually open, but I guess they decided to close it after that:

https://www.ecma-international.org/publications/standards/Ec...

bt8482• 7 years ago

What would you do with a totally open specification?

doikor• 7 years ago

Read it?

nitrogen2• 7 years ago

It's easier to recover corrupted media if you know everything about the underlying format and protocols. As an example, I was able to recover a partition that was partially overwritten by just a few sectors by finding a backup copy of the superblock and writing it into the correct place armed only with hexdump and dd. That would have been impossible without detailed info about ext2/3/4 available.

Also look at the floppy imagers people have made, that operate at the magnetic flux level.

nitrogen2• 7 years ago

Just noticed your username. Any connection to the TV decoder chip of the same name?

Mave83• 7 years ago

If you only save your data as a single copy on a single tape, then the price is quite low (around 1/3 of Glacier). However you can reach prices of below Glacier using online storage with GB/s performance and access times of milliseconds using Ceph with the right hardware selection including every cost. You can learn more about how from croit.io.

raws• 7 years ago

In terms of longevity MDisc is king (1k years estimate). How does tape compare to drives and ssds without power? and what kind of environment is required? Someone mentioned losing 5k$ worth of tapes to temperature hike.

meshr2• 7 years ago

I can confirm that they are extremely unreliable compared with other hardware. There is also no support from vendors.

Youden• 7 years ago

MDisc is not king. Their DVDs were outperformed by competitors in a test done by the French National Archives [0].

Further, DVDs are effectively obsolete these days and MDisc's Blu-Rays have no scientific studies backing their claims to longevity compared to competitors like the DVDs did. One of their largest advantages relative to DVDs is completely gone in Blu-Rays as well: The inorganic dyes and recording layer (these are now standard for essentially all BD-Rs).

[0] https://www.lne.fr/sites/default/files/inline-files/syylex-g... and full test and results in French: https://francearchives.fr/file/de7f8ea96ceb4ce38eb6d9278b3df...

Dylan168076• 7 years ago

you can buy cheap LTO-7, reformat them to M8 and get 9TB of native storage

Note: You can't reformat them this way. This is a permanent choice you make with a blank tape.

mmaunder3• 7 years ago

Also IIRC you cant do this yourself with a regular LTO7 tape and drive. I think you need some kind of drive shuttle or you need to buy the tapes preformatted by someone who has the right gear. I have an LTO7/8 drive and I cant do this.

mister_hn• 7 years ago

sure, but they are really slow. It depends on your use cases.

adrian_b22• 7 years ago

Tapes are slow only at the latency of random access. Their sequential transfer speed is lower than that of good SSDs, but higher than that of HDDs.

hoseja• 7 years ago

Cookie consent, adblock warning, autoplaying video, some kind of newsettler form that I just glanced at before closing the tab.

What a miserable website.

aspyct• 7 years ago

Got JavaScript disabled (thanks firefox and noscript), and I have none of that bullshit.

Give it a try :)

moviuro• 7 years ago

Even just the "Reader" view of Firefox fixes that: https://support.mozilla.org/en-US/kb/firefox-reader-view-clu...

Delfino• 7 years ago

Yes, I am an IB Computer Science teacher and we still teach tape drives as a form of storage so I was excited to share this story with my students based on the title. Then I went to the website. No way I'm sending a link to a site like that to them.

randomchars• 7 years ago

Try it this way: https://outline.com/hDqg78

Generally, you can add outline.com before any news or blog domain, and you get a reader view to share.

NohatCoder2• 7 years ago

Note that the article is straight up lying about the price of HDD capacity, 12TB drives generally cost more per TB than lower capacity drives, the best capacity deals are generally in the range 3 to 8 TB. Comparing cheapest to cheapest, tapes only go for around half the price of hard disks.

adrian_b22• 7 years ago

Normal HDDs are at least 3 times the price of tapes.

To reach only double price of tapes you must choose some super-ancient junk, e.g. 1 TB 3.5" HDDs, which are much slower than tapes and which require a huge size and weight for the same capacity as tapes.

12 TB HDDs have about the same volume as LTO-7 tapes (i.e. about the same volume as two 6 TB tapes), but they are much heavier. LTO-8 tapes have double capacity compared to LTO-7.

mxfh• 7 years ago

Dunno, if shucking is an option, external consumer WD drives are regularily available under EUR 200 for 10 TB, while bulk drives of same capacity or higher for dedicated storage use are at least 50% more expensive. In my experience this is a releatively new development, that those two markets diverged that much. 2015 and before bulk HDD where usually cheaper than external ones.

If you do care about warranty you can just keep them in the external usb casing and enjoy having an abundance of power supplies and USB 3 cables.

If it's 24/7 hot storage it also uses 50kWH per Year at 6W. That's also about 15EUR/Year minimum energy costs per drive.

Me feeling is, that all HDD manufacterers are extracting as as much value from the market as they get away with (meaning having more of a smell of informal price fixing, then of free market) while they still can, before they will be made osolete to solid state storage in 5 years latest.

NohatCoder2• 7 years ago

If we compare them to tapes, it doesn't seem fair that the hard disks should be kept spinning, booting them up on demand still gives you vastly lower latency than tapes.

Me feeling is that the competition in the tape market is also a bit rusty, $3000 for a fancier VCR, what a steal! ;-)

jammygit• 7 years ago

I was told once that certain types of storage medium will demagnetize over time and that you need to spin them up to prevent data loss

jart• 7 years ago

If you want to talk about MBps, imagine a truck full of tape cassettes driving from the Google data center to off-site storage.

perl4ever2• 7 years ago

From the sound of your comment, you haven't heard of this:

https://aws.amazon.com/snowmobile/

...putting stuff on a big truck is a very viable alternative to using a series of tubes.

segfaultbuserr5• 7 years ago

I'm well-aware that tape is still the king for datacenter-level storage, and I'd love to have a tape drive for my ever-increasing size of my personal backup.

Their advantages to me is obvious: its inexpensive price and great longterm reliability beats any alternative choices: hard drives, optical drives, or solid-state drives, and sometimes cloud. Also, it's cool to put my "tape archive" (.tar) files on a real tape! But I found for personal uses, tapes aren't cost-effective, the entry-cost and initial investment of the tape drive is extremely high, it's all enterprise-grade SCSI or SAN. Unless you already have tens of terabytes of data (video studio?), buying a tape drive is unjustified. HDDs are still sometimes the only cost-effective choice for personal use.

Now I mainly use HDDs for large backups (> 300 GiB), multiple Blu-ray disks for medium-size backups (< 100 GiB), and use cloud for the smallest archives. I don't know if Amazon's "mail your drive to the cloud" service is available at my location, but it's also an attractive solution.

I'd like to hear about your experience.

Moru• 7 years ago

Did you try backblaze.com yet?

segfaultbuserr5• 7 years ago

I think any cloud solution is limited by the broadband uplink. It takes 24 hours to upload "only" 200 GiB of data over a 20 Mbps uplink. It's fine for longterm backup that literally nobody touches unless the computer breaks, but one still has to be patient enough to backup a RAID array...

stiray• 7 years ago

The tape story is really killing me. I even bought one old tandberg for 200 euros, used and run it on an ancient hardware just for backups. I just dont get it that in all those years not a one single vendor decided to make less expensive tape devices for the consumer market.

EvanAnderson• 7 years ago

There used to be less expensive consumer-oriented tape drives (floppy-based QIC, various parallel port drives, VXA, an DDS/DAT). That segment of the market has been subsumed by backup services, USB-based solid state or hard disk-based storage, and writable optical media.

acuozzo• 7 years ago

not one single vendor decided to make less expensive tape devices

https://www.youtube.com/watch?v=TUS0Zv2APjU

lmilcin• 7 years ago

I think this is a bit misleading. Tape is not just the tape cartridge itself. Quality drives are very expensive and so is tape management. Managing access to tapes at scale is also not trivial and complicates a lot of your processes.

Another important factor everybody seems to forget is that you don't want the tape to be your only backup. They are so slow to restore that you typically only use tape as last resort.

For my personal backups I calculated a bunch of regular 4TB drives, external USB 3.1 bay which takes 2 of them at once and an off-site storage (place where I can keep drives) to be best solution.

mcv• 7 years ago

Exactly what I was thinking. Tapes may be efficient while you're not accessing the data on them, but data you're not accessing is ultimately useless. You need the infrastructure to restore from backup, and if you've got so much data backed up that the cost savings of tape really matter, you might have to backup (and restore) so much data that the lower speed also really matters.

op00to2• 7 years ago

Tape drives are consumable. LTO tape is abrasive, this cleans the heads but also wears them down. I used to manage a scientific computing facility with a PB scale rape archive. We ingested maybe 50 TB a day to tape, and would chew through one of the 8 drives at least twice a week.

ummwhat• 7 years ago

I know this is a typo, and I don't usually point out such things, but maybe fix "rape archive"

segfaultbuserr5• 7 years ago

Tape drives are consumable. LTO tape is abrasive, this cleans the heads but also wears them down.

Is it possible to replace the RW head on a tape drive? I guess the manufacturers generally don't want this to be easily replaceable, so they can sell more drives...

q3k2• 7 years ago

Was this same generation drive and tape? I've only seen this be this bad when doing MP (LTO4/LTO5) reads/writes in BaFe-targeting drives (LTO6/LTO7).

These old MP tapes are like sandpaper to new drives.

andyzweb• 7 years ago

I'm curious for more details. Are these full-height or half-height drives?

one2zero• 7 years ago

Was this "rape" archive at the Vatican perhaps?

rowanG077• 7 years ago

Are you using green media? Green media is very abrasive and it's recommended to burn them in before usage. This can literally saves you thousands upon thousands in tape drives.

time0ut• 7 years ago

Ya, this is an odd article. It seems aimed at the consumer. Maybe it's a lack of imagination on my part, but I can't think of a consumer use case that calls for tape. I think the article could have benefited from spelling one out.

archi42• 7 years ago

I was considering to mirror my cold storage backup (12 bay NAS with a few hours uptime per week) to tape for long term archival of private data (e.g. photos) or stuff on the internet I think is worth having a backup of (e.g. the http://unrealarchive.org).

But the tape storage requirements and danger of running into trouble with the reader (especially considering I could get a misaligned one when buying used, and if that one breaks => tough luck) convinced me this isn't a good choice. [edit] For the same money I can just get a bunch of additional disks, and use two for simple, redundant, offsite backups; and the remainders to swap out old disks in the NAS pre-fail.

Porthos9K• 7 years ago

Do manufacturers even offer consumer-grade tape backup solutions nowadays? Do tape drives even use USB, or is the tech still using SCSI? I can't seem to get a straight answer from Google, but I doubt it.

dathinab• 7 years ago

The interesting thing is that for long term high reliable cold storage (for e.g. research data) using a bunch of tape robots seems to still be quite viable idea. (The robots read, check and replace broken tapes automatically getting data from tapes can take long so you normally put some form of sdd cache in front of it.)

Then you place the tape robots into a fireproofed room with exhaustive protection measurements (e.g. filling the whole room with CO2 or similar on fire alarm, having extra fireproofed doors which are comparable with bomb shelter doors etc.).

apocalyptic0n3• 7 years ago

Out of curiosity, where are you keeping them off-site? I have about 15T of used storage right now. I have it in a RAID and then backed up to another RAID in my home, but I am missing the third piece of the backup puzzle. Are you just putting them in a safety deposit box? Or do you put them in a colo of some kind?

Accujack• 7 years ago

For personal backups (unless you're the king of personal Big Data) tape doesn't make sense in most cases.

Where tape shines is for offsite copies, DR copies, or duplication for safe keeping of any kind, not backups.

Tape will work for backups, but you have to structure your working processes around tape (serial access) to the backed up data. If for you "backups" includes rapid access to an old version of a file you edited or deleted by mistake and pulling in data that's too old for you to want to keep on expensive disks so you can process it, tapes aren't a good option, for reasons of access time.

Most modern IT professionals would use NAS or SAN snapshotting to handle the first case above, where you did an oops and need a copy of the file as it was before that. A snapshot directory on a NAS usually gets implemented using a journaling file system or some other mechanism for maintaining a point in time view of the system, so you're not keeping another complete copy of your data, only journal entries listing what changed when.

For the second case where you have too much data to fit on your expensive SSDs and you might need to use some of it e.g. next week or next month, what you want isn't tape, it's a cheaper hard drive. IT types usually call this "near line storage". State of the art spinning hard drives are a good option, but if those are still too expensive you can go a couple generations of technology back and get lower price/TB. The problem you run in to doing this is that you need more physical space and hardware and power/cooling for the lower capacity drives.

Tape is great for when you want to keep data but may not need to access it ever or might need to access it months from now, and when you can accept a delay in access of a day or two. DR copies, offsite vaulting of data for legal purposes, logs from systems you want to keep in case you ever need to check back on what happened 5 years ago, bulk data from experiments that may never be accessed again but which could be critical for someone if it's needed (and may therefore turn out to be extremely valuable)... those are the scenarios for tape, and the price per TB stored is lower than anything else.

The company where I work off sites data for long term storage regularly, including a copy of our main ERP database. We meet legal compliance requirements the cheap way by copying stuff to a tape we can access and paying someone to store it for us. Backups of production systems are done to disk, to a NAS or SAN block storage device that is then copied to tape if needed.

Edit: Forgot to mention the ERP databases' size... it's in the Petabyte range, it runs the entire company. If we tried to transmit it to a cloud backup provider, without spending a giant amount of money on a high speed internet connection we'd never complete a daily backup before it was time for the next one. On top of that, if we need 2 copies of it the storage price gets very high very fast - cloud storage is very convenient, but costs more than even local spinning disk. Tapes,on the other hand have a small incremental cost if we already have the drives and robot, and we can make as many copies as we like by paying only for the tapes and the time.

cf141q53253• 7 years ago

Does anyone have any experience with encrypted tapes and could give a few words about their setup? Mainly other options then the manufacturers encryption.

adrian_b22• 7 years ago

I do not trust any kind of encryption inside devices, because I cannot verify how the keys are managed. The essence of encryption is that the key must not be stored together with the encrypted data, but if a key is ever used inside a device you cannot be sure that it can no longer be retrieved from it in the future.

The data files that I am writing to the tapes have their metadata written into a database to be able to retrieve them in the future, then I group them in chunks of an approximately fixed size (50 GB in my case), then I compress them, then I encrypt them and then I add some redundancy (with par2) for error recovery. A simple solution for compression with encryption is the lrzip program, but for more flexibility in encryption you could use e.g. the openssl program invoked with the desired parameters. There are many encryption programs and any of them is better than using the encryption option of a tape drive.

So I queue the files to be written on the tape in a directory and when the size of the directory exceeds a threshold (50 GB in my case) a script is invoked that does the processing mentioned above, including the encryption, then it writes the result on the tape. A similar script does the inverse operations when I retrieve a file or group of files from the tape, giving its position as a tape number + a tape file number as obtained from the file database.

cf141q53253• 7 years ago

Thank you for the detailed answer.

meshr2• 7 years ago

This is not true at least in my experience. Price doesn’t tell you about reliability. I have 10% of broken cartridges and 66% of broken tape drives. These devices have mechanical parts and are sensitive to magnetic fields so they could be damaged easily. If you don't believe me then I can send you all this garbage to you

ai_ja_nai• 7 years ago

Assuming that you already own the tape recorder, WTF

unnouinceput• 7 years ago

For me there is only one factor that matters in backup. Not restore/backup speed, not price per byte, not fashion - only reliability. Does the backup medium will perform without corruption when times come to read it? That's the only thing that matters.

I had this problem - store and forget, long time, as in decades. Tried a lot of stuff and: Personal conclusion - offline backup sucks, regardless of medium. Tape, harddisks, zip drives - demagnetization of medium. CD, bluray - scratch and you're done. Thumb stick, SSD, non-volatile memory in general - needs a plug-in once in a while. So what I use is encrypted volumes and upload to cloud...and a dedicated SSD that I have on me at all times if I need offline access (which gets plugged in daily).

Erlich_Bachman5• 7 years ago

How are you gonna scratch your optical discs? Do you move them around all the time? Also with something like Rar recovery records or or other redundancy measures, a scratch does not mean that you are "done", because the data can be recovered anyway (with storage size penalty, but setting aside 10% of storage for being able to recover from random scratches is well worth it).

masklinn3• 7 years ago

How are you gonna scratch your optical discs?

IME the problem of optical discs is that unless they're molded (ROM) they degrade over time.

masklinn3• 7 years ago

For me there is only one factor that matters in backup. Not restore/backup speed, not price per byte, not fashion - only reliability. Does the backup medium will perform without corruption when times come to read it? That's the only thing that matters.

Only if the other parameters are within certain bounds, otherwise your backup medium would be macro-scale physical engraving on a highly resistant substrate, at least some sort of stone, ideally a refractory or noble metal. Rhodium would probably be your ideal standard.

bmrp2• 7 years ago

Thumb stick, SSD, non-volatile memory in general - needs a plug-in once in a while.

What do you mean by this? It is non-volatile after all.

adrian_b22• 7 years ago

Any non-volatile memory is characterized by a "retention time" after which it may loose the information stored in it.

The retention time depends on the storage conditions, especially on temperature & humidity. For ancient flash memories, with large cells and 1 bit per cell, the retention time can be of 10 years or even 20 years. After that the capacitor will discharge, because the insulation is not perfect.

For modern flash memories, with much smaller cells and wit 3 or even 4 bits per cell, the electrical charge variation that will lead to wrong bits is much less, so the retention time can vary from a few years for a new SSD stored at low temperature to a few months for a used SSD stored at a higher temperature. At a high enough temperature (not impossible to reach, e.g. when forgotten in sun light), the retention time can be reduced to days.

When a SSD is used, i.e. it is powered, the microcontroller inside it rewrites its cells from time to time to avoid losing data. If you store it without using it, after the guaranteed retention time you may lose data. Consumer SSDs and memory sticks are guaranteed for a retention time of 1 year, so you should not expect more than that.

hyperpallium• 7 years ago

If you overshoot demand on one performance metric, buyers shift to another as yet unsatisfied criterion.

e.g. 8" HDDs are also cheaper per byte than 5.25", than 3.5", than SSD's.

learnstats2• 7 years ago

"But there's something else that tape offers that no other storage medium currently offers and that's on-the-fly, transparent compression which can go up to 2.5:1"

I don't understand this point. Why would tape allow compression that other forms of media don't?

apolog• 7 years ago

Not an expert here, but could it be because you can actually slow down the tape 2,5x and still be readable?

makapuf2• 7 years ago

I think what is unique here could be the "on-the-fly, transparent".

But this kind of compression can be better achieved I think with one core of your CPU, plaintext compression of 3x shouldn't need lzma (and if you really need to compress 22TB of plain text with no binary or multimedia files, well maybe you could compress it slightly with lz4 or a simple zlib compression before sending it).

tenebrisalietum• 7 years ago

Lots of tape drives offer built-in compression. The drive firmware does it. On AIT3 drives you can enable or disable it.

Compression may have been a big deal in the ages of sub-1Ghz single core systems but not anymore. Especially since you ought to be encrypting before writing anyway.

Why does it continue? Probably so they can double the capacity that they put on the tape label, which is always the compressed capacity and not the native or real capacity. Plus backward compatible with such encoded tapes.

Spooky23• 7 years ago

People are very conservative about backup, and some backup solutions don’t compress in software.

FrozenVoid2• 7 years ago

Every week without fail i see articles shilling tape storage. Do people realize only really Big Data benefits from tape? Unless you write multiple petabytes of data that you rarely need each day, tape is inferior in every way to hard disks. A huge corporation that counts every cent and needs lots of cold storage might benefit from tape, and companies still prefer better storage ( https://engineering.fb.com/core-data/under-the-hood-facebook... ).

"Good enough for Google, good enough for you"(a lie that neglects economies of scale that force Google to use tape). https://www.overlandstorage.com/blog/?p=323

adrian_b22• 7 years ago

No, tapes become preferable to HDDs at less than 100 TB written, not at multiple petabytes. For many applications and for many people, including for myself in my home, it makes absolutely no sense to use HDDs for anything, but only SSDs & tapes. For long term storage tape is superior in every way to HDDs, while for fast access SSDs are superior in every way to HDDs.

Obviously, because the drives are extremely expensive, whoever needs no more than 20 or 30 TB of storage cannot afford tapes and must use a few HDDs. The best storage technology changes in time. 20 years ago I was using optical disks and 10 years ago I was using HDDs, but since I went back to tapes, like I was previously using 30 years ago, I have saved a lot of money and I have much less worries about the HDDs becoming defective.

FrozenVoid2• 7 years ago

How does it compare with 8TB USB3 drives(140$)? https://pcpartpicker.com/product/YpJtt6/western-digital-elem...

tapehead• 7 years ago

I can’t agree with your assertion that “tape is inferior in every way to hard disks”. Tape is cheaper per TB than HDD; why put data on more expensive media if you don’t have to. Tape is also more reliable by orders of magnitude than disk. You are more likely to be eaten by a shark than suffer an unrecoverable error using LTO-8. Tape is more durable and portable than disk. It also requires no power at rest and can be used to create an air gap to thwart cyber attack and malware. Finally, tape areal densities for tape are increasing far more quickly than for disk due to superparamagnetism, which makes tape far better suited to storing archive data in the IoT era of Big Data. Tape is less ideal if you have very stringent RTO or an application that needs fast random access to first byte. But in most cases, for its core strength of longer term data archiving, tape is better than disk or cloud (although as you note, the irony is that tape underpins major cloud vendor archive tiers). Don’t be absolutist; use the right storage, for the right purpose, at the right tape.

ryanmercer• 7 years ago

It never ceases to amaze me how much tape I still see imported, mind you I see way more SSDs, then HDDs but still a fantastical amount of tape.

stelonix• 7 years ago

Are there any cheap tape drives? If not, what's the issue? Is it the fact the prices are kept high because it's used mostly by business/enterprises customers? If so, are there any hopes of LTO becoming mainstream for home desktop usage? Couldn't find much info on Google about this.

NeedMoreTea• 7 years ago

DAT (Well DDS) used to fill that niche. Used to be pretty reasonably priced - for tape. Went for a few hundred a drive, an order of magnitude less than LTO. Smaller capacity each generation though, but is an older standard.

Disappeared for no apparent reason about a decade or so ago.

craig1f• 7 years ago

This is the principle behind AWS Glacier

not_a_cop75• 7 years ago

And why wouldn't it? Tapes as far as I know cost pennies to manufacture. It's easily over 90% plastic in most cases. The low quality ingredient on a low end tape is probably still ferric oxide - not exactly an expensive chemical if I know anything about chemistry. Perhaps cobalt is still heavily used as part of the coating as well.

bufferoverflow• 7 years ago

That's if you don't count the extra time required to deal with tape backups.

idlerboris• 7 years ago

It's crucially important to take managing and lifetime of such storages into account. I doubt it can fight with SSD or HDD.

scraft• 7 years ago

We had a Ditto drive and a series of tapes, bought circa 1996. We used the software it came with on Windows to perform full back ups of the machine, cycling between a set of tapes (maybe 6). Backup would complete overnight. Never had to restore from it. It was a consumer friendly, easy to use product and cheap. My father was an accountant that ran his own practise, so needed to know his files were safe. That went from carbon copies into a second filing cabinet, to 5 1/4" floppies for groups of clients, to 3.5" floppies, to Ditto tapes to USB sticks to Microsoft Cloud storage (which sinks between laptop and PC). The oldest days of filing cabinets and floppies were easy, the middle days of Ditto and USB storage was more worrying, and the final days of cloud storage went back to being simple again. Now he is retired so none of it matters.

This was all donkeys years ago, so it is interesting to hear how things have changed.

vonseel• 7 years ago

As someone totally unfamiliar with tape aside from knowing some places still use it for archival and recording audio to tape - something totally different - is incredibly expensive and most (if not all?) recording tape is out of production, I’m surprised storage tape is still a thing.

How does storage tape even work? Is this literal tape in a cartridge much like tape in a cassette tape? I assume we’re not talking about data stored in 1s and 0s? Is this technically an analog medium!?

Going to do some googling now to quell my curiosity.

The article does seem targeted at consumers. I’m going to guess I’d need some expensive hardware to even use one of these $60 tape drives.

Tape still beats SSDs and hard drives when it comes to price per byte

Related Stories