Online backup to S3 for the Mac

http://www.haystacksoftware.com/arq/

by sreitshamer • 16 years ago

60 52 16 years ago

This page could really benefit from a screenshot or if you're feeling adventurous, a screencast.

I don't want to commit to downloading this app without seeing how it functions. I understand it lets me back up to S3, but how?

I dare say if you add these images, your conversion rates will skyrocket.

Assuming this is your site, anyway.

sreitshamer14• 16 years ago

You're probably (hopefully?) right. It was on my list, but maybe I'll move it to the top of my list.

bugs• 16 years ago

Most definitely, I for one don't try many new programs without some sort of screenshot (if you do a screencast also put screenshots up as not everyone can always watch a video or wants to watch a video).

iamcalledrob2• 16 years ago

Just adding a followup, you've added screenshots and I'm now considering purchasing your app.

It worked for me :)

quizbiz• 16 years ago

If I do not have S3, what would be the total effective cost?

sreitshamer14• 16 years ago

It doesn't work without an S3 account. The total cost is $29 (one time) plus monthly S3 charges. For example, 33GB backed up at S3 would cost $4.95/month in storage charges, plus some (relatively small) transmission and transaction charges. Upload transmission to S3 is free through June 2010.

jwr• 16 years ago

Good thing somebody brought up the problem of encryption with other backup providers (like Backblaze).

Still, problem is that if you want to back up a lot of data, S3 quickly becomes rather expensive. In my case, storage costs alone for around 550GB that I currently store on Backblaze would run me $82.50 a month. Not exactly small change.

Willie_Dynamite3• 16 years ago

Sounds nice. Just too bad the contents of my NAS would cost $700 a month to keep on s3.

samt• 16 years ago

+1 on this. Any decent NAS software (FreeNAS, OpenSolaris, etc) with an iscsi target will work great with the (free) GlobalSAN initiator and Time Machine.

notmyname• 16 years ago

True, but one big "gotcha" to this is that there is no way to do a complete restore from a time machine volume. There is no iscsi initiator on the OS X system DVD (and therefore no way to select your TM backup for a complete system restore).

sreitshamer14• 16 years ago

I was thinking of adding Pogoplug support. That way storage would be much cheaper, although backing up to a Pogoplug at a friend's house may be much less reliable than backing up to S3.

Would anyone be interested in this?

Willie_Dynamite3• 16 years ago

I'd be far more interested in a generic ssh target thingy, than support for specific peripherals.

pclark3• 16 years ago

Pogoplug support would be great. Especially if I can limit the bandwidth on the device and leave it at a friends house. In fact, I'd leave it at work.

mtigas• 16 years ago

The price scaling issue worried me at first, and then I noticed the monthly budgeting — great feature. In the long run, you’ll always sacrifice price (vs. an external hard drive) for off-site data persistence anyway.

In any case, this looks really awesome for documents/projects backup to supplement my TimeMachine drive.

sreitshamer14• 16 years ago

Right, I use it as a supplement to my Time Machine drive as well, for my most important stuff (documents + photos).

Maybe I should call out that use case on the landing page.

pclark3• 16 years ago

I can't judge how much it'll cost to back my iMac up. I have about 750Gb of data.

Willie_Dynamite3• 16 years ago

You can't figure out what 750 * 0.15 comes to?

joezydeco• 16 years ago

Remember after June 30th it's also (750 * 0.15) to upload it.

Frazzydee• 16 years ago

Perhaps he is saying that the per-gig cost is not clear. It says buy now for $29, but the extra $0.15/GB/month to use S3 is hidden under "Why use S3?" (you might not expect costs to be there)

We both know what S3 is, and know that they charge per gig. But the market of people who want online backups for their macs don't necessarily know this.

halostatue• 16 years ago

Lots. Roughly US$112.50 per month. CrashPlan and SpiderOak are better price value, although I might look at using this to have a secondary backup of critical items. Even that, for me, though, is about 200GB (photos & music) or about $30/mo.

rarrrrrr• 16 years ago

I'm a bit biased, but I don't feel S3 is the best possible fit for a frugal consumer's backup data. S3 is a high performance, general purpose solution capable of very low latency.

Having that performance makes it more expensive than an archival specific solution, which tolerates latency just fine as long as high throughput is maintained.

There's a reason why Mozy, Carbonite, SpiderOak, etc. all run our own data centers. I can sell a 100GB account for usable across unlimited computers for $50 per year and be profitable.

That said, the market certainly has plenty of room for more backup choices, and any option to help people avoid data loss is a win. An exclusively Mac app seems like a good idea.

sreitshamer14• 16 years ago

I agree. For me though, backup comes down to trust. I trust Amazon to provide the S3 service reliably. I find it harder to trust Mozy, Carbonite and the rest, partly because they don't even bother to restore my Mac files' metadata correctly.

mikepurvis2• 16 years ago

I don't know how other people do it, but certainly for me, I have things that are at different levels of importance to me. And a lot of the most important are not really all that big, nor are they changing particularly frequently. I wouldn't pay a premium of $100 a month just for the convenience of having everything backed up in this super-redundant, universally-accessible way.

I would happily use a product like Arq (or Time Machine) for my documents and current photos, to make sure the backup is always current, and then whenever the backup hits 50gb, move it over to an off-site drive (at work, a relative's house, even in a safety deposit box).

matthewer• 16 years ago

Has anyone tried both Backblaze and this?

sreitshamer14• 16 years ago

I have. I chose my own encryption password, but in order to restore files, I had to enter the password into the Backblaze web site. Backblaze used it to decrypt my files on their server and assemble them into a zip file on their server. They then sent me email saying I could download the zip file.

Also the Mac metadata weren't restored properly: http://www.haystacksoftware.com/arq/backblaze-backup-bouncer...

mikebo• 16 years ago

I had Mozy and I used their built-in encryption to backup ~250GB. I went to restore some wedding photos while at my parents' house, so I downloaded their decryption utility, and started the decryption of a 10gb subset of my backup.

Only about 1/3rd of the files decrypted correctly. The rest of the files were totally corrupted -- not cool. I contacted support and we investigated the problem for ~3 months. Luckily I didn't have dataloss, but what exactly was I paying for here?

We never got the problem resolved and I ended up getting a full refund for 1.5 years of service. Make sure to test your backups no matter where you store them.

cperciva4• 16 years ago

what exactly was I paying for here?

You were paying for not worrying about whether you would lose your data. And that's exactly what Mozy gave you... at least until you discovered the corruption. :-)

mstevens• 16 years ago

This has just reminded me I've got a lot of data backed up with you and no idea if the restore works :)

KirinDave• 16 years ago

I'm confused how this product can charge so much more than JungleDisk but not really do anything different? Is there a difference I'm missing?

sreitshamer14• 16 years ago

Well, JungleDisk is a monthly service from $2-5 month (plus S3 charges) depending on the plan you choose. Arq is a 1-time $29 cost (plus S3 charges). The other big difference is that JungleDisk is a Java app and doesn't have a native Mac look and feel.

Also Arq is a lot simpler to use. I really need to put up a screencast to show that.

anotherperson• 16 years ago

From the Carbonite Backup Bouncer test:

One of backup-bouncer's tests is called "combo-tests" where it tests several different file metadata types on the same file. Carbonite didn't restore this file at all.

Instead, Carbonite left a file called "Carbonite_Restore_F161_G1.tmp". It's a 1-line text file that reads "gotta boogie".

cperciva4• 16 years ago

As the author of Tarsnap, I'm a bit biased, so I'm going to refrain from comparing the two; but I have one question for the author: Aren't S3 per-request fees a problem? One of the reasons I have all Tarsnap data go through the Tarsnap server is to allow me to "bundle" blocks together and amortize per-PUT costs.

sreitshamer14• 16 years ago

The per-request fees ($.01 per 1000 PUTs and $.01 per 10,000 GETs) aren't a problem for 2 reasons:

1) Arq packs small files (under 64k) together into "packs", a bit like git does. So instead of thousands of PUTs of tiny files (which would also be very slow) it does fewer larger PUTs.

2) Arq caches indexes of those pack files locally, and caches the list of larger S3 objects as well. So it doesn't have to do a GET to figure out whether an object with a given SHA1 is already in the S3 account. (Arq stores every object by its SHA1 hash, like git does).

The file format is open (it's your data after all): http://www.haystacksoftware.com/arq/s3_data_format.txt

cperciva4• 16 years ago

Arq packs small files (under 64k) together into "packs"...

Ok, so if you have lots of 64 kB files they're all PUT separately, using 15625 PUTs/GB (i.e., $0.15/GB)? I guess that's not too bad.

The file format is open

Nice! Good to see someone who believes in openness. (How about the source code, so that people can confirm that you're doing what the spec says?)

One thing caught my eye glancing through that document, though: "The name of each blob is its SHA1 hash." That's the SHA1 hash of the encrypted data, right? (If not, it definitely should be!)

towndrunk• 16 years ago

Honestly, I don't mind the extra fees as long as I can restore successfully. I just want a solution that works for backup and restore with out issue.

sunchild• 16 years ago

Looks good. What about workgroup admin?

PStamatiou• 16 years ago

I have been using Arq for the last 3 weeks. It's very basic software at the moment, but it does the job. It lets me see file revisions and restore files as necessary. Unfortunately these files can't be seen directly in S3 due to the packing nature of how they're stored.. so if your mac crashes and you want a file, you'll need to find another mac and install Arq to retrieve your files.

sreitshamer14• 16 years ago

Right. The files are named using the SHA1 hash of their contents (to enable de-duplication) and small files are packed together (to dramatically improve performance). I thought that was the best solution given the limits of the S3 API. JungleDisk does something similar I think.

What sort of alternative for restoring would you be interested in? Maybe an open-source command-line utility?

zacharypinter• 16 years ago

A command-line utility is always a big plus for those who like scripting. Maybe a FUSE script/app/plugin?

rshields• 16 years ago

Is there some way to make the app not suck up all the bandwidth on my teeny weeny cable modem?

sreitshamer14• 16 years ago

Yes, you can set a maximum transfer rate. You can also choose "automatic transfer rate" which slows backups when you're using the Internet interactively.

Online backup to S3 for the Mac

Related Stories