Ask HN: Using S3 for personal data backup?

https://news.ycombinator.com/item?id=1707832

by ryanteo • 16 years ago

10 29 16 years ago

news.ycombinator.com

Hi HN, I would like to ask whether anyone has experience using S3 for personal data backup (documents, photos, video) and work.

Background: Just experienced an external hard drive failing on me. Traumatic and irritating =)

Hope that you can share how you've set it up (rsync?), some examples of the data you've stored and the costs that you are paying each month.

Luckily for me, Amazon just opened a data centre in Singapore =p

Thank you! Ryan

dotBen2• 16 years ago

CyberDuck (free, for Mac) otherwise JungleDisk ($$) to connect your machine to S3.

Don't backup your data to Singapore even if you are local, it will cost you more (compare costs across S3 DC's) and you don't need backup data to be close to you in the network.

S3 can work out expensive if you want to host a lot of data. Sometimes it can be cheaper to buy one of those 'unlimited' backup accounts with Mozy/etc as after a while the cost will be cheaper than Amazon.

amock2• 16 years ago

The EU west region is just as cheap as the US standard, so if you want to backup off continent and not pay more you can. As far as the unlimited backup plans go, how does their durability compare to S3 or SpiderOak? It seems like if they're so much cheaper they can't have as much redundancy.

dotBen2• 16 years ago

It seems like if they're so much cheaper they can't have as much redundancy.

They probably don't have the same amount of redundancy as S3 but then redundancy is a subjective thing.

A backup service might claim that your data is only stored in one place (ie no redundancy) but that it's very stable.

Amazon S3 might be more diverse but it's also volatile - Amazon claim that they cannot guarantee not to loose data.

cperciva3• 16 years ago

I'm using S3 for personal data backup.

How I set it up: http://www.tarsnap.com/

g0atbutt• 16 years ago

Doesn't the person that created tarsnap post here?

amock2• 16 years ago

Maybe you already know this and are making a joke, but cperciva is the creator of tarsnap.

cperciva3• 16 years ago

Yes, I think he does.

mahmud• 16 years ago

I hope that was nod to the putnam story, otherwise WOW, Colin must get 'em all the time.

ryanteo3• 16 years ago

Thanks! This might be something I'm looking for.

Ryan

appl3star2• 16 years ago

Come on - at least you could say you are the developer. Otherwise thats not fair. (BTW We use tarsnap as a server offsite backup.)

vitovito• 16 years ago

bradfitz's brackup supports securely backing up data to S3 (and elsewhere), including incremental backups.

http://code.google.com/p/brackup/

I don't actually use this, I just followed its development. For my backups, I have a script that spins up an Amazon EC2 instance, mounts an EBS disk and does an old-fashioned rsync of my entire machine.

This does mean I pay for the entire disk all the time every month, plus the EC2 time, and it's probably slower and more expensive than an equivalent brackup, but I also didn't have to think about it.

Right now, the EBS disk isn't bootable, but I could make it so and in theory just boot up an EC2 instance and immediately have my entire machine at the state it was last backed-up as.

briandoll• 16 years ago

I used to use S3 to backup data on my mac mini via JungleDisk. I was paying somewhere around $10-15/mo in storage costs after the initial fee of uploading the bulk of the data. Unfortunately after a system upgrade (not sure when or what specifically), the automatic sync w/ JungleDisk just stopped working. For several months none of my data was being copied back to S3.

What I really need is simplicity that _just works_. I found that in Dropbox and haven't gone back since. I use it to back up ~100Gb of data. On my other machines I just choose not to sync the larger directories and everything works perfectly.

BerislavLopac• 16 years ago

I absolutely, positively second DropBox. It integrates seamlessly, and you may have 100GB for only $200 a year.

But th best thing is that you can try it for free: it gives you 2GB initially, and an additional 250MB (up to 10GB) for each new user you refer. (E.g. if you apply via this link, I'll get an additional 250MB: http://www.dropbox.com/referrals/NTIxMzc2ODk _hint hint ;)_ )

ryanteo3• 16 years ago

Thanks! I've been introducing dropbox to friends too (not just because of the referral=) ) It just works. However, I was wondering whether it's cheaper to use S3 instead, since I intend to store around 200 GB worth of files.

appl3star2• 16 years ago

Cut these bloody referal links.

And I would -not- recommend DropBox for critical backup files. First, they can read all content. Second, they got some problems with large folders or system directories.

anarchitect• 16 years ago

I started using JungleDisk after a I got my laptop AND bankup disk stolen a few years back. It's worked well for me.

epynonymous• 16 years ago

i use http://www.mozy.com

foundbobby• 16 years ago

I just set up s3fs on my linux server (http://code.google.com/p/s3fs/wiki/FuseOverAmazon) and pointed it at Google Storage for Developers (http://code.google.com/apis/storage/docs/getting-started.htm...). It's intended for use with S3 as the name implies of course. It's been working flawlessly so far :)!

hellweaver666• 16 years ago

Isn't that basically just what Dropbox does?

hga• 16 years ago

I rather like http://rsync.net/. My critical, need to keep it offsite data backup requirements fit it well, there's one simple monthly charge and I can use rsync plus a handful of blessed shell commands to manage it. They also figured out a way to support git, which I use for projects that are too incomplete or unpolished to put up on GitHub. I looked very hard at S3 but it's rather inflexible for exactly what I want to do.

That said, I'm only storing my parent's photos and videos of their grandchildren (I'll lose all my media if my apartment is truly wiped out by a tornado, then again I'm unlikely to survive that). If you need to backup a lot of stuff S3 will be quite a bit cheaper (for me either is pocket change as of now).

CatalystFactory• 16 years ago

My friend runs: http://www.syncplicity.com/ - might be of use.

irv• 16 years ago

another vote for tarsnap - can't complain really. i knocked up a simple Haskell app to manage the process (I'm learning Haskell, so comments and suggestions welcome) if you're interested @ http://github.com/irv/tarsnap-backup

metalwheaties2• 16 years ago

Do you know about http://www.wuala.org ? It's free if you share your disk space, it's redundant, and it's fast and easy. Backups, rsync, and regular file storage via a browser in a Java based client (runs everywhere I have tried) make this universally good. I never paid a cent to them, shared a couple of machines worth of 50GB of extra space that is up 100% of the time, and I have > 100GB of space available to me.

Linux backups are really easy: Wuala's client uses fuse to mount itself as a file system, so you can simply do periodic tar, or you can let their own backup mechanism back up your stuff to a special folder they maintain perfectly. You can get to your backed up files anywhere in the world via Wuala.com. This feature has saved me gobs of effort a bunch of times already.

metalwheaties2• 16 years ago

BTW, I have no affiliation with Wuala. I just like it.

ryanteo3• 16 years ago

Thanks!=) I'll look in that... Did not know there were so many options available.

rphlx• 16 years ago

Consider using duplicity to backup to gmail IMAPS. 7GB in a luxury, sea-side GOOG datacenter, free of charge forever. Need >7GB? Create another gmail acct.

The problem with paid services is that if you have even a temporary credit card/billing problem, they may nuke your data. Also, they're probably less reliable than gmail.

cpswan• 16 years ago

I've been using www.SMEstorage.com for this. They support multiple storage clouds as back ends, and have DropBox style tools that give me automated sync between multiple machines (and mobiles). Keeping all of my 'work' data safe and in sync between my desktop and laptop is costing me cents a month.

Ask HN: Using S3 for personal data backup?

Related Stories