Show HN: Raw Image Storage for photographers using S3 and Glacier

by jjbohn • 13 years ago

44 49 13 years ago

Looks good, I signed up. You only support photos?

jjbohn6• 13 years ago

Right now it's photo centric just to make sure we get that experience down. Eventually I could see it move to other areas though. My wife's a photographer and I saw a need for this kind of service that was focused on photographers. There's lots of other photo backup, but the only ones that do raw files are more syncs than anything. She needed a place to persist images she was done with in a cheap way.

photoGrant• 13 years ago

I work for an advertising photographer. We constantly work with 3 local (seperate) copies of the same data. Would love to find a viable cloud service.

If we could test this that'd be great!

StavrosK10• 13 years ago

If someone open-source, encrypted backups at that price, I would buy it in an instant. What I want is, basically, rdiff-backup on EncFS, but I haven't found a way to hack the two to work together.

eli• 13 years ago

Tarsnap too expensive?

StavrosK10• 13 years ago

Around 8 times more expensive than this, yeah.

jewel• 13 years ago

I have a friend that backs up his photos to my computer using a combination of ecryptfs, rdiff-backup, and rsync. It goes something like this:

    mount -t ecryptfs ~/.photos-backup-crypted ~/photos-backup
    rdiff-backup ~/photos ~/photos-backup
    rsync -a --del ~/.photos-backup-crypted remote-server:photos-backup

StavrosK10• 13 years ago

That's pretty much what I want, but it requires you to have as much free space as the thing you want to back up.

What I wanted to do was mount EncFS in reverse mode (so it gets plain files and creates an encrypted virtual volume) and rdiff-backup that to a remote host, which works very well. However, the problem is that, when your directories are disparate, you can't easily make them appear as one directory, so, to back up 5 locations you'd have to do this 5 times.

I can fake it using AUFS or similar, but it's too large a dependency. Someone wrote an EncFS patch to let it follow symlinks in reverse mode (https://code.google.com/p/encfs/issues/detail?id=147), which would solve the problem, but I don't think EncFS is maintained any more.

darkarmani• 13 years ago

SpiderOak provides encrypted backup for any types of files for around $0.09/GiB. When i signed up for a free account, i got a 10% off email a few days later.

StavrosK10• 13 years ago

I currently use SpiderOak, but I'm nothing if not paranoid, and they aren't open-source. Other than that, it's pretty good for backups, and quite cheap, too.

slashdotdash2• 13 years ago

* Arc http://www.haystacksoftware.com/arq/

* JungleDisk https://www.jungledisk.com/

Arc supports encrypted backup to Amazon S3 and/or Glacier, works with occasionally connected external hard disks perfectly. I use this to backup RAW photos from Aperture stored on an external USB hard disk (at least 50GB worth).

StavrosK10• 13 years ago

Very nice, thank you! It looks like neither supports headless Linux, though...

davak• 13 years ago

Can you describe your workflow for aperture?

brucehart• 13 years ago

Have you tried Duplicity (http://duplicity.nongnu.org/)? I use it for backups and think it's great. You can even backup directly to Amazon S3.

StavrosK10• 13 years ago

I have. Unfortunately, due to the way it encrypts files (it basically turns everything into one big, encrypted tarball) it needs to redo the whole thing from time to time, as diffs stack and it gets unwieldy.

Combined with a slow home connection, it basically never managed to upload all my files, and it could never resume. That's why I want to go the EncFS route, I don't care if file sizes and numbers are visible if it makes uploading resumable, sane and easier.

davidroetzel• 13 years ago

There was a story on HN a few weeks ago by a photographer wanting this kind of service. I found the discussion very interesting.

Is this in any way inspired by that? Or is this a coincidence?

jjbohn6• 13 years ago

Very much inspired by it. Wish I could find the post actually. I had some services running for my wife's photography business, but nothing that could really be released in any way. Seeing that other post though made me start taking the steps to turn it into a SaaS.

josephb• 13 years ago

There was some good discussion in this thread:

https://news.ycombinator.com/item?id=5673628 see orofino

And in an even older thread:

https://news.ycombinator.com/item?id=4619132 see akmiller

tekseven• 13 years ago

I believe the article you are looking for is here: http://paulstamatiou.com/storage-for-photographers

Discussion: https://news.ycombinator.com/item?id=6020969

Greatri• 13 years ago

That's what I love about HN :) The minute I read the post I wondered how long it would take for someone to build it - turns out it takes 29 days.

vadvi• 13 years ago

"With our pay-as-you-go pricing, you pay a base rate" LOL

benjamincburns2• 13 years ago

Couple of things...

First, I really dig the "Made with ♥ in Nashville, TN" footer. Way to have some pride. People need to pimp their non-SV residences more.

Second, I've been mulling over doing something like this. Since you're doing it already, please steal my idea. Make a connector for LightRoom and make it stupid simple to link a RAW with exported JPEGs.

xvrl• 13 years ago

If you're looking for Lightroom integration, mosaic (http://mosaicarchive.com) provides an almost identical service and already has a plugin.

benjamincburns2• 13 years ago

But was mosaic made with love in Nashville? ;-)

richardkmichael• 13 years ago

I do IT support for a few working architectural photographers. Over the years I've thought about this too (at least, I've worked the numbers a few times). It's hard to make it work; they need a minimum of 8TB of storage for existing material, then growth by about 2.5TB yearly (to date, but most are about to start time-lapse video). Though, your cold storage approach is intriguing.

Napkin calculations:

  0.04  /GB * 1000 = 40.00 /month (curr) =  480.00/yr
            * 2000 =                     =  960.00/yr
            * 3000 =                     = 1440.00/yr

  0.004 /GB * 8000 = 32.00 /month (cold) =  344.00/yr + 48.00/yr/TB
                                                      + 100.00/yr/2TB
                                                      + 175.00/yr/3.5TB

  1 shoot = 25GB (stills only) or 50GB w a 30 sec TL (add 25GB)

  ~100 shoots /yr ==> 2.5TB/yr @ 25GB, 5TB/yr @ 50GB

[Aside, that's 5-12 mins to upload (25Mbp/s - 10Mbp/s, only 3 mins @ 50Mbp/s). So consider additional internet cost.]

Without frequently juggling current to cold, or doing yearly compounding calculations, I'd eyeball this at ~$1300/yr, with growth of $150/yr, each year.

One way to make this more attractive is to leverage the online storage to build other recurring-revenue generating services, but most photographers don't go for that type of thing. (In my experience, stock image sites, etc. are primarily the realm of hobbyists.)

You also have the trust/confidence problem. I suspect every photographer will want to maintain their own backup as well. So, they won't "save" money (even if online storage was less expensive, but I suspect it's not) because they still need to pay for local storage. Of course, for apples to apples, you need to factor in time to maintain those backups, either my hourly rate or their own; things like disk-testing, data juggling (downsizing 1TB -> 3TB drives, etc.). But again, it's still 'extra' cost, because they won't (IMO) switch to solely online storage.

A question:

Are you providing tools to allow photographers to leverage their online storage? E.g. Dropbox-like sharing with clients (even just contact sheets?), integration against online print services, store-fronts for image sales, etc.?

All that said, I'm interested in seeing your private beta though. I think this can work, one day. :)

jjbohn6• 13 years ago

Definitely want to let them leverage the storage eventually, including contact sheets, psd storage, etc. Starting with just the basics for now to see where it goes.

Paul123455342• 13 years ago

I don't use Smugmug but what possible advantage does this have over Smugmug? Even their top tier plan with lots of features is only $300 a year. All their plans have unlimited storage. $300 a year would only cover 625GB at $0.04/GB.

dagw• 13 years ago

Smugmug only backs up the jpegs you upload for display. If you want to store your RAW files on smugmug you have to use their separate smugvault service which costs $0.09/GB/month

Paul123455342• 13 years ago

ahhh ok, I didn't notice that :) I stand corrected

deanclatworthy• 13 years ago

I found your pricing to be unclear. You say it starts from $0.04/GB, but is this for the cold storage or active storage. If I uploaded 50GB of photos to "cold storage" immediately, what would be the monthly cost? What if I then uploaded another 10GB into active storage, what would the monthly cost be for 50 cold and 10 active?

I've signed up though. Email is similar to username ;-)

jjbohn6• 13 years ago

Yeah, I'm a pretty terrible copywriter. Let me see if I can make it more clear on the site. Thanks for the input.

mieses• 13 years ago

I've been using the FastGlacier GUI client and paying $0.01/GB/mo. Of course Glacier pricing increases for retrieval, early deletion, etc. But there is only one middle-man (Amazon). On OSX, Arq looks like the best client.

geoka92• 13 years ago

If you're content with the 2048x2048 pixel limit for your stored photos, picasa is ulimited and free. (You do have to sign up for Google+; without it, the pixel limit for photos that don't count towards the quota is smaller.)

Groxx2• 13 years ago

And if you're content with JPEG only (I assume it's not RAW, though I have no real way to test it).

Though tbh I've begun recommending Flickr over Picasa. Better (significantly) 3rd-party app support, 1TB is more than the vast majority will use in a long time, and it's easy to default everything to private if desired.

geoka92• 13 years ago

I find working with Flickr more cumbersome if you only use the web UI. For example, Flickr has a limit of 200 photos that you can upload in one batch, which is a problem when you have a slowish upload link (ADSL/cable).

tghw• 13 years ago

We applied to YC with this idea last year. I wrote about it in the last thread on this topic:

https://news.ycombinator.com/item?id=6021507

Overall, your two biggest challenges are going to be convincing people they need backups (they still don't know, we tried) and pricing. Currently, Backblaze gives me unlimited backups for $5/month or less. With nearly half a terabyte of photos, it's cheaper for me to back up everything with them than use a service like this, which doesn't back up the rest of my stuff.

That said, I hope you guys can find a way to make it work!

jjbohn6• 13 years ago

Great posts. Thanks for sharing them. Will definitely keep all of what you said in mind.

Show HN: Raw Image Storage for photographers using S3 and Glacier

Related Stories