Replacing Dropbox in favor of DigitalOcean spaces

https://mitjafelicijan.com/replacing-dropbox-in-favor-of-digitalocean-spaces.html

222 193 5 years ago

What kind of digital ocean plan did you get for your setup ? I've been thinking about putting together something similar but the price to get a reasonable amount of space seemed prohibitive.

mitjafelicijan14• 5 years ago

I choose Spaces which costs $5 a month. And you don't actually need VPS for this.

https://www.digitalocean.com/products/spaces/

__soter__• 5 years ago

$5 for 250GB of storage and then $20 per TB ($0.02/GB) after that seems a bit steep.

  Plans and Pricing
  The base rate of a Spaces subscription is $5/month and gives you the ability to create multiple Spaces.
  The subscription includes 250 GiB of data storage (cumulative across all of your Spaces). Additional storage beyond this allotment is $0.02/GiB. If you cancel your subscription by destroying all your Spaces, your bill will be prorated hourly.

https://www.digitalocean.com/docs/spaces/

secfirstmd4• 5 years ago

Interesting comparison price when something like Sia is so cheap per TB (4 dollars). Will be interesting to see how it developers.

mitjafelicijan14• 5 years ago

It looks they also have S3 support now. Will need to try it out.

https://blog.sia.tech/introducing-s3-style-file-sharing-for-...

jrockway• 5 years ago

I think $20/TB*month is exactly what GCP charges, and is probably on par with what Amazon charges.

StavrosK• 5 years ago

I tried to use Sia once but I needed to download the entire blockchain. Not a fan of storing 5 GB locally so I can store 1 GB remotely...

foolmeonce2• 5 years ago

I think hetzner is about half an expensive (€8 tb/m) and I personally prefer ssh/rsync to http/S3.

ogre_codes3• 5 years ago

I don't know what Hetzner is, but if rsync is available, it's a much better option, particularly if you can do incremental syncing you can get a fairly robust setup with very little code.

gruez2• 5 years ago

I think hetzner is about half an expensive (€8 tb/m)

What offering is that? Their "storage" page quotes 26 euros for 5TB.

christophilus3• 5 years ago

I’ve been happy with Wasabi.

mynegation• 5 years ago

With prices like that, Microsoft Office 365 subscription that gives you 1TB of OneDrive space, seems like a better deal.

dyeje• 5 years ago

If you're cool with the spartan interface, why not use S3? Isn't most of the value prop on DO Spaces the CDN component?

mitjafelicijan14• 5 years ago

With s3cmd you could use it with almost anything supporting S3. I work with AWS every day and am a bit shell shocked by it :) And for personal things I prefer DO. And my needs are low. My whole sync folder is about 5gb.

But yeah, I could use AWS S3. I agree.

quaffapint• 5 years ago

You could also go the route of using Backblaze for storage and Cloudflare for delivery. You'll pay $1.25 for the same 250gb ($0.005/gb) and get free bandwidth through CF.

geonic• 5 years ago

Can you expand on that? I store my backups on B2. Are you saying I can download all data via cloudflare without paying for B2 egress?

cweagans• 5 years ago

Info is here: https://www.cloudflare.com/bandwidth-alliance/backblaze/

22c• 5 years ago

Interested in this myself. There appears to be some sort of partnership between Backblaze and CloudFlare

https://help.backblaze.com/hc/en-us/articles/217666928-Using...

https://help.backblaze.com/hc/en-us/articles/360010017893-Ho...

secfirstmd4• 5 years ago

This is cool

2Gkashmiri2• 5 years ago

Does this have a limitation? Size or whatever ?

I want something to store peertube videos and serve them. This sounds like too good to be true

ddorian432• 5 years ago

It is, you didn't step on a free video cdn. Cloudflare probits using video unless you're in Enterprise.

bithavoc• 5 years ago

Cloudflare limits uploads to 100MB per HTTP request, download bandwidth is pretty much free.

Dylan168072• 5 years ago

Though it's always worth testing based on use case. I use B2 for backups, which works great. But when doing some testing with a more interactive load, I really didn't like how many timeouts I got.

ShakataGaNai• 5 years ago

If you're doing this more "quick and easy developer" style, Backblaze is definitely something worth following up. B2 would be way better for the cost ($0.005/gb). DO Spaces, Linode Object Storage and AWS S3 are all about the same price ($0.02/gb).

jph• 5 years ago

You wrote about deplatforming-- heads up that DigitalOcean does deplatform people. I have direct firsthand knowledge of DigitalOcean deplatforming a security disclosure professional reporting a root breach bug.

For your secure backups, I can suggest Tarsnap by Colin Percival, who's a security expert and a frequent contributor on Hacker News. I don't get anything for recommending Tarsnap; I'm just a customer. https://www.tarsnap.com/

mitjafelicijan14• 5 years ago

Damn! Thanks for the info. I don't get these companies. That person should be thanked and not deplatformed.

mitjafelicijan14• 5 years ago

This is really awesome! I will definitely try this out. Thanks again.

ev12• 5 years ago

DO will also use "machine learning" and "AI" to shut down your machines or terminate your account if your usage pattern is disliked.

mitjafelicijan14• 5 years ago

Has this been happening already? I have been using this script for about 4 months now, and I had no problems yet. I do syncing once a day.

christophilus3• 5 years ago

They flagged my account and pulled 100% of my servers with no mitigation for 12 hours until I convinced them I was legitimate. DO still handles that incredibly poorly, which is the reason we no longer host anything critical there.

mitjafelicijan14• 5 years ago

Well, this is a discouraging news. Where have you moved your critical operation?

Shoop• 5 years ago

For clarity, is "They" DO or tarsnap?

PenguinCoder• 5 years ago

What or who do you suggest instead ?AWS?

cghendrix• 5 years ago

I’ve been hearing about this happen for a long time. Unfortunate that it’s still the case.

freedomben2• 5 years ago

I had a terrible, terrifying experience with DO where they hard locked my account for days (and it would have been weeks if I hadn't made a huge stink) because of a misunderstanding. They ban/lock first, ask questions later, and that's not cool and I can't have my (and my customer's) important infrastructure treated that way.

I now use Linode for most of my stuff, and once they have cloud firewalls available in Dallas I'll be moving the rest of infrastructure over as I can.

I really need to write up my story into a blog post I can link to, but until then, I put a recap here a few days ago for anyone interested in the details. Mine wasn't as bad as some people's, but pretty scary personally: https://news.ycombinator.com/item?id=25806086

WarOnPrivacy• 5 years ago

They ban/lock first, ask questions later

Given the sheer volume of dodgy packets that originate from DO network space, I find this surprising.

dannyw• 5 years ago

I’ve had only positive experiences with Linode, including incidents where I faced absolutely massive DDoS attacks.

On the other hand, my account is suspended from DigitalOcean, I have multiple droplets they refuse to give the data to, and they even had the audacity to keep billing me until I did credit card chargebacks. I still have no idea why I was banned.

Maybe because I used the GitHub student pack promo but I had already finished uni? I don’t know.

azinman23• 5 years ago

My issue with tarsnap vs Dropbox is entirely workflow. Right now I can treat my Dropbox effectively as an external drive that happens to live on my file system by making everything selectively synced. If I want something backed up, I just drag it into that folder. But once it’s backed up I can turn on selective sync and free up my disk space.

Backup of personal data is often a 1-way endeavor — here’s a dump of photos I scanned that I don’t need to look at anytime soon. But with tarsnap I can’t do that, nor can I then have the ease of browsing to the file and just opening it while it transparently downloads on demand.

TheSmiddy• 5 years ago

If you're transfering data to a backup service and then deleting it locally that's not backup, that's just cloud storage.

ajmadesc• 5 years ago

How do you make them sync?

0x4265776172653• 5 years ago

I don't get anything for recommending Tarsnap; I'm just a customer. https://www.tarsnap.com/

From their site:

"The Tarsnap service is built on top of the solid platform provided by Amazon Web Services."

markdown• 5 years ago

DigitalOcean does deplatform people... I can suggest Tarsnap

Tarsnap runs on Amazon.

qmmmur• 5 years ago

This seems insanely expensive compared to Restic combined with backblaze

Saris2• 5 years ago

Am I reading that right, that 1TB of storage used would be $256 per month + bandwidth costs?

autoditype• 5 years ago

I just did the same calculation and I that looks right

AnonC4• 5 years ago

You should read HN posts from the past where @patio recommended that tarsnap should raise prices even more.

arsome2• 5 years ago

As tarsnap is run by a single person, it seems hard to suggest it to serious businesses due to the inherent risks there as well as the extremely high storage costs and difficult cleanup procedures compared to competing tools. I tend to prefer solutions like restic + B2 for their price and support, where you at least have a larger company behind the storage backend, you could even directly use it with S3 and it'd still be cheaper than Tarsnap.

Decent write up on the cryptography in restic here: https://blog.filippo.io/restic-cryptography/

Some tarsnap issues detailed here: https://www.kalzumeus.com/2014/04/03/fantasy-tarsnap/

jclulow• 5 years ago

Oh my word, that article on tarsnap is a frightful bore. I was a tarsnap customer and I thought it was great. If Colin is happy running it more like a utility than a rapacious VC-backed hyper-growth racket, I'm not sure why that is bad.

The only reason I stopped being a customer was because another rapacious racket of a business, Comcast, introduced miniscule but enforced data caps in my area, so online backups aren't attractive anymore and I've gone back to external drives and offsite rotation. When I cancelled, Colin sent me a personal e-mail to make sure it was alright to delete my backups. It was probably the best exchange I've ever had with a service provider.

fletchowns2• 5 years ago

Unfortunately restic was a no go for me due to not being compatible with B2 keys that only have the permissions readFiles,writeFiles,listBuckets,listFiles (no deleteFiles). I don't want the attacker to be able to delete any backups if the manage to get to the B2 keys.

I believe this is the ticket that would add support for this to restic: https://github.com/restic/caddy/issues/2

SamReidHughes2• 5 years ago

Tarsnap seems to have become a multi-man operation.

https://github.com/Tarsnap/tarsnap/commits/master

varispeed2• 5 years ago

When a business becomes serious? Do you need to hire several workers to look legitimate? If that's still not enough, do you need to make yourself a slave of VC? Only problem I can see is that one man operation is susceptible to the run over by a bus scenario. However that is still better than a woke VC asking the "owner" to cancel some users.

LordAtlas• 5 years ago

Wait, their pricing is $0.25 per GB per month, so $250 per month for 1 TB? What the heck?

cowsandmilk• 5 years ago

The price is after tarsnap deduces and compresses, which it is quite good at and costs for 1 TB of real world data are much lower.

https://www.tarsnap.com/deduplication-examples

Barrin92• 5 years ago

what's the advantage of tarsnap over say, doing an encrypted backup with borg to my gdrive folder? I've never heard of Google random deleting encrypted blobs from people's storage and it's way cheaper

mherrmann• 5 years ago

I used to use Tarsnap and am still stuck with it in several projects. I wish I weren't. Restoring backups takes ages. If you need to restore a production server in an emergency, and the whole server remains down for an hour because that's how long Tarsnap takes to restore some tens of Gigabytes, then you will very quickly look for alternatives. I raised this issue with Colin, Tarsnap's creator, multiple times. And the answer has always been "maybe some day". I now simply encrypt backups myself and store them in an S3 or Linode bucket. Much faster. For personal backups, I love restic syncing to Backblaze, where I back up 1.4 TB or so for $5 per month.

mythz2• 5 years ago

$0.25 /GB is pretty expensive. I'm using Hetzner Storage Box [1] which has tonnes of connectivity options where 1TB is only 9.40€ (works out at 0.0094€ /GB)

[1] https://www.hetzner.com/storage/storage-box

AnonC4• 5 years ago

Yes, Tarsnap is really quite expensive if you have data that doesn’t get deduplicated well.

MrDresden2• 5 years ago

I've never heard about Hetzner before, but those are some good prices.

Have they been around for long, and you know of any external public audit that has been done on them?

AnonC4• 5 years ago

I would actually compare DigitalOcean to a Facebook or a Google or an Instagram. You’re banned or blocked and cannot reach a human to find out why or what can be done. DO support is just a bunch of auto mailers sending the same template emails, in my experience.

varispeed2• 5 years ago

That's why so far best idea is to go bare metal with couple of different providers and sync data to multiple servers. In case one provider decides to cancel you, you can still have your data in another place. You can also have a copy at your own server. To have access to all machines as if they were on a local network, you can use tinc-vpn or for convenience ZeroTier.

patrec• 5 years ago

Colin Percival is obviously a smart and very skilled guy but I am mystified why people on HN keep recommending tarsnap, it seems a terrible product for almost any imaginable audience.

If I'm a normal end user, I will probably get a vastly easier to use product at about 1-10% of the price from Backblaze. And if I'm a serious business that can easily afford the > 10x premium and engineering to configure the backup I probably wouldn't want to entrust it to some company with what looks to be a bus factor of one and the apparent technical limitation that a restore might cost me a few DAYS of unanticipated downtime [1].

I mean, sure, if you are some unix nerd wanting to backup your dotfiles and a few small documents (or repos) for maybe around ~$50/year, why not go with some artisanal backup service for HN street cred (and a laudable open source donation policy)? But what other good use cases are there?

Am I missing something obvious?

[1] https://news.ycombinator.com/item?id=25621093

JoachimSchipper• 5 years ago

E.g. relative to the author's script, tarsnap get you automatic deduplication across backups... which, if you're like me and basically generate more and more data, allows you to store lots of historic backups at basically no overhead over just storing today's data.

Also, good encryption and security; e.g. Tarsnap is trusted by Stripe (https://www.tarsnap.com/testimonials.html).

Tarsnap does have real downsides - restores can indeed be slow, bare per-GB cost is high, and "like tar" is not a user interface that everyone will like - but there are definitely upsides, too.

pronoiac• 5 years ago

Note, it's not bidirectional sync right now:

All I need to do is write a Bash script that does the reverse and downloads from remote server to local folder.

mitjafelicijan14• 5 years ago

True!

All I need to do is reverse parameters.

Documentation states:

s3cmd sync LOCAL_DIR s3://BUCKET[/PREFIX] or s3://BUCKET[/PREFIX] LOCAL_DIR

But I need to test it out before I post that solution.

muxator• 5 years ago

I wouldn't trust that script too much: there is no error checking at all.

What happens when s3cmd fails and after two months you discover the "Vault sync succeded" emails you have been getting were all illusory?

If the author is reading: please take the time to update your example, including proper error checking (at least "set -eu"), otherwise the people on the internet that are going to copy your script are in for a hard time.

Skeleton of the "incriminated" script:

  #!/bin/bash
  [...]
  s3cmd sync ...
  [...]
  notify-send [...] "Vault sync succeded [...]"

mitjafelicijan14• 5 years ago

I agree with you. I should add a disclaimer that this is still being worked on.

I will add what you proposed.

Thanks for your input.

lanstin• 5 years ago

Always start all your bash scripts with

set -e

You'll never regret it and might be very very very happy it was there. And if you start with that, then as you work on the shell script, you'll be more likely to make the script be idempotent (checking for files before copying, checking for lines in files before appending/sed'ing etc.) Idempotent >> non-idempotent for "alter the state on an end point" things.

gtirloni• 5 years ago

I haven't looked at the code but the author would benefit from using shellcheck.

https://www.shellcheck.net

notyourday3• 5 years ago

Whyyyyyy? Just use rclone.

asidiali2• 5 years ago

This is really cool! Thanks for sharing - going to give it a go.

Similarly, I took a stab not too long ago at attempting to replace my Dropbox [1] - I opted for an implementation using Git and FSWatch.

Pro: version control for your files

Con: more expensive and takes more storage

[1] https://sidia.li/simple-dropbox-alternative/

mitjafelicijan14• 5 years ago

Love it. Your solution is much more polished and Dropbox like. Very nice!

asidiali2• 5 years ago

Thanks for checking it out - may we both never have to succumb to third party syncing platforms again!

mciancia• 5 years ago

Another clickbait, it's a replacement for Dropbox in the same way as a tent is a replacement for a car - you can sleep in both.

Whole thing just looks like poor ad for DigitalOcean

mitjafelicijan14• 5 years ago

I never claimed that this is 1 on 1 replacement for Dropbox. Dropbox is much much much more than what I am doing here. And I agree with you on that point. I choose DO because it is very easy to set up and get started with.

But for my needs this is all that I need and it solves my use case.

And regarding DO, its just one of the possible S3 providers out there. One could choose something else.

paxys2• 5 years ago

There is a lot of open source file sync software out there (ownCloud, NextCloud, Seafile, Syncthing, ...) that would be much better suited for a self-hosted Dropbox replacement than a single bash script. You are many orders of magnitude more likely to lose your data due to a code bug or unstable network than deplatforming.

I looked into setting up something of the sort for myself a little while ago, but unfortunately the pricing just isn't feasible if you have a large amount of data (say a few hundred GBs). You could easily end up paying more than $100/mo for storage + compute + bandwidth, while Dropbox and others charge ~$6-$10/mo for like 2TB.

homedrive• 5 years ago

You can host Nextcloud at your home using something like NextcloudPi+DDNS (https://nextcloudpi.com) or HomeDrive (https://homedrive.io, which has ngrok-ish end-to-end tunnels integrated), and the monthly cost would be almost $0. :)

whoopdedo• 5 years ago

I recently set up Nginx on a DO droplet as a reverse proxy to my home server over Wireguard. It's not $0 but it doesn't directly expose my home IP. And I have an authoritative resolver on the LAN so local clients go directly to the private address.

pmontra• 5 years ago

It's cheap if you already own the hardware. Even $300 could be 50 months (4 year) of a remote server at the prices suggested by some of the posts in this page.

Furthermore self hosting a backup server at home doesn't protect against burglars and fires. You'll probably lose both your data and their backup.

That's why I prefer a local backup plus a remote one (a combination of git repositories and file storage. )

clvx• 5 years ago

Exactly. I use Nextcloud with Spaces as a backend. It’s been sitting there for months by now.

jamiesonbecker2• 5 years ago

Instead of an instance, you could just rent a dedicated server with some mirrored TB of drives.

For example, https://www.server4you.com/dedicated-server has 2x 2TB for $25/mo in St. Louis, MO and unmetered bandwidth.

OVH Cloud in the U.S. has servers with 32GB of RAM and also 2x2TB drives starting at $59/mo and unmetered bandwidth. (both of these are French companies that are expanding into the U.S.) If you are interested in other locations (including Canada), check out ovh.com, kimsufi.com, and soyoustart.com (all part of the enormous OVH global footprint).

If you're in EMEA, or don't mind an extra 100 or so millisecond (SSH works fine), check out OVH and Scaleway (France), and of course Hetzner (Germany). Each of these offer dedicated servers for very reasonable prices, and copious amounts of bandwidth with reasonable pings to almost anywhere.

For example, https://www.kimsufi.com/us/en/servers.xml currently has 500GB hard drive servers with Intel Atoms available for $5/mo in both Canada and France, or bigger hard drives for just a few bucks more. This is OVH's cheapest line of servers, but OVH does a pretty extensive burn-in test before deploying them to you, and they tend to be pretty solid. (keep backups in case a hard drive fails, but that's always a good idea anyway.)

secfirstmd4• 5 years ago

I've found Contabo to be even cheaper and despite what some have said, haven't had any issues with them.

cbozeman• 5 years ago

I'm a big Hetzner dedicated server fan. I've been using them for 5 years after switching from OVH. Its astounding the amount of hardware and transfer they offer for a extremely reasonable price.

noizejoy• 5 years ago

For example, https://www.kimsufi.com/us/en/servers.xml currently has 500GB hard drive servers with Intel Atoms available for $5/mo in both Canada ...

It says “currently being replenished” for the 2 lowest price packages in Canada

curben• 5 years ago

Hetzner managed nextcloud costs $12/month (https://www.hetzner.com/storage/storage-share?country=us).

ww520• 5 years ago

Yes. Been using Syncthing and it's been great. Only down side is there's no iOS port.

marcus_holmes2• 5 years ago

Another Syncthing fan here. The software is great. But I used DO spaces for storage and it's pricy. One of my "to do" lists for 2021 is find a cheaper storage solution and move my backups over.

Maakuth• 5 years ago

Yep. It probably works quite well even between desktop, laptop and Android devices, but personally I have a home server running one instance. This allows me to always have the latest copies of files available for any of my devices and that's where the backups live too. Probably a Hetzner box or something should be quite cost effective way to have the same benefits.

cwmartin• 5 years ago

I heard about this from another recent HN thread https://www.mobiussync.com however I haven’t had a chance to try it yet so I can’t attest to its quality.

mythrwy• 5 years ago

Wasabi is S3 compatible storage and costs $6/TB per month.

NextCloud and ownCloud can be set up to utilize Amazon S3 or Wasabi storage. This works really well and is very inexpensive. Both my GFs business files and my personal/work files all are backed up Dropbox style for less then $10 a month including web and mobile access.

I found NextCloud a little buggy compared to OwnCloud but it has more features and unfortunately I'm stuck with it for the time being and can't easily switch back to NextCloud. The sync works fine though.

ies7• 5 years ago

My little nitpick: Wasabi has that 90 days minimum storage policy.

swjz• 5 years ago

Dropbox does a great job syncing a large number of small files -- I believe they compress the small files somehow before uploading to the cloud. And plenty of other optimizations that make me happy to pay Dropbox's premium. Otherwise, Google Drive and OneDrive are cheap alternatives too.

jamesmunns• 5 years ago

I actually really like DO spaces, I use it for hosting a small audio podcast. One downside is they give you NO way to see your bandwidth usage, other than seeing whether you got a bill overage or not. It was pretty surprising to me that they HAVE the information for billing, but don't expose it to the user of spaces in any way.

jmuguy• 5 years ago

I also found that to be really bizarre and there’s threads of users requesting usage info going back to over a year ago.

midasz• 5 years ago

If you're just doing simple stuff where bandwidth matters I'd look into OVH/kimsufi, it's cheap unmetered dedicated servers.

AnonC4• 5 years ago

Could you please expand on how you host a podcast and what software you use (or point to resources that can help)? I’m assuming you mean hosting the podcast audio files, not just RSS feeds to the episode files stored elsewhere. Thank you.

centimeter• 5 years ago

High efficiency and very high performance trustless backup setup: send incremental (encrypted, compressed) ZFS snapshots to rsync.net, hetzner, or buyvm. This produces smaller backups than rsync or similar tools, produces them faster, and also shields your data from prying eyes, as it’s all encrypted when it leaves your machine.

tarruda• 5 years ago

Rsync.net also supports borg backups at a cheaper price than ZFS: https://www.rsync.net/products/borg.html

notyourday3• 5 years ago

If that's what you are doing why not just go to Wasabi? It is cheaper, storage is the only thing that they do and downloads is free until the GB transferred is under GB stored per month.

tra3• 5 years ago

I've never heard of them -- how long have they been around?

chromatin• 5 years ago

2017

bachmeier• 5 years ago

I looked at Wasabi not long ago.

For customers that use Wasabi’s pay-as-you-go pricing model, Wasabi has a minimum monthly charge associated with 1 TB of storage ($5.99/month). If you store less than 1 TB of active storage in your account, your total charge will still be $5.99/month (plus any applicable taxes). See FAQ#4 for more details.
With Wasabi minimum storage retention policy, minimum number of days are as follows:
90 days (default) for customers using Wasabi’s pay-as-you-go pricing model

Wasabi's not intended for small-scale use cases.

Dylan168072• 5 years ago

Six dollars a month isn't small scale?

notyourday3• 5 years ago

It is six dollars a month per TB billed on a GB level with 1TB minimum, with no weird plan tiers with S3 compatible API and no stupid per API call charges, reasonable pull pricing ( either free or cheap ), not related to other accounts.

What, exactly is that level of control of $ spent ( as it is going to be lower than pennies ), that people want while maintaining reasonable reliability, stability, durability and scalability of storage.

ogre_codes3• 5 years ago

When I read "Replacing Dropbox", reading about a 1 way sync system for backing up a single system isn't what jumps to mind.

The magic of Dropbox is that it sync files across multiple computers and allows multiple people to share files. This doesn't do any of that.

You can use Dropbox for that, but there are definitely cheaper/ more robust solutions.

contravariant2• 5 years ago

Well to be fair the remote backup is the part that's hardest to recreate with a self-hosted solution.

ogre_codes3• 5 years ago

Is it? I used to do something similar with a couple rsync commands. In fact I think the rsync solution is more robust because you can create incremental backups.

0x4265776172653• 5 years ago

I have a 50TB+ NAS in my garage rack. Which I'm not utilizing completely. I wonder if there is a market for renting out some of my storage for someone else's encrypted backups. Something like a website, where I can list my price/gb & connection speed -- or maybe people could swap. Maybe I want to store 5TB off site and I'm willing to let someone else store their 5TB on my server. The website would act as the middleman for introducing the two customers.

ukd1• 5 years ago

Siacoin and Filecoin are basically this; you can store small parts of files under contracts that give you crypto in return. Occasionally you'll get tested by the network to check if it's online, and you have the files you should have. A good use of crypto, though last time I looked the ROI wasn't great as a host.

0x4265776172653• 5 years ago

That sounds more taxing on my connection than what I was thinking. Not really up for 100 connections, but if I'm matched up with 1 or 2 other users then that would seem ok.

paxys2• 5 years ago

The amount of liability you'd be taking on would make it absolutely not worth it for 2 customers and a few dollars a month in income.

On the opposite end, I'd trust Google or Dropbox with my encrypted backup much more than a rack in someone's garage on a home internet connection.

jiggawatts• 5 years ago

I went down a bit of a rabbit hole of Digital Ocean and their "security" for production workloads.

Show me any other vps provider that silently provides access to customer A's data to customer B after receiving commands from customer A to destroy their instance and then I'll believe you guys aren't at the very bottom of the "takes security seriously" list.

From: https://github.com/fog/fog/issues/2525#issuecomment-31337481

YC News Discussion: https://news.ycombinator.com/item?id=6983097

You do not need to scrub or write anything to not provide user A’s data to user B in a multi-tenant environment. Sparse allocation can easily return nulls to a reader even while the underlying block storage still contains the old data. ... On top of all of that, when I pointed out that what they were doing was absolute amateur hour clownshoes, they oscillated between telling me it was a design decision working as intended (and that it was fine for me to publicize it), and that I was an irresponsible discloser by sharing a vulnerability.

From: https://news.ycombinator.com/item?id=20091026

You've got an additional problem though, which is that this tells us you have two support channels: one that doesn't work (i.e. yours, the one you built), and one that does (Twitter-shaming). The first channel represents how you act when no one's watching; the second, how you act when they are. Most people prefer to deal with people for whom those two are the same.

From: https://news.ycombinator.com/item?id=20064169

Speaking of randomly locking accounts, the post-mortem kills me:

The initial account lock and resource power down resulted from an automated service that monitors for cryptocurrency mining activity (Droplet CPU loads and Droplet create behaviors). These signals, coupled with a number of account-level signals (including payment history and current run rate compared to total payments) are used to determine if automated action is warranted to minimize the impact of potential fraudulent high-cpu-loads on other customers.

From: https://www.digitalocean.com/blog/an-update-on-last-weeks-cu...?

In other other words, DO will kill your account with a curt email staring simply: "We have reviewed your account and have declined to activate it. No further information or action is required from you." for simply using "too much CPU"! https://pbs.twimg.com/media/D76ocofXoAY_xB5.png

mitjafelicijan14• 5 years ago

I would pin this post to the top if I could. Thanks for this info!

marcinzm• 5 years ago

Is there anything commercial that is both cheap and popular that doesn't cut corners to get there?

edit: Point being, I don't see how DO could get their act together as their business model essentially requires them to act like this. If they don't cut corners in one area then they will somewhere else.

chrissnell2• 5 years ago

I must say, I concur with this person’s assessment of the Goog-exit feeling. I ditched GMail for Fastmail and my own domain last week. I had been putting this off for years because it seemed like such a massive pain in the ass and I was afraid that I would just end up with two email services.

In the end, it took me one evening to update every service that I cared about to use my new address. The rest (mostly e-commerce companies) I’ve been doing as I use them.

nsomaru• 5 years ago

Been thinking seriously about this.

Do you just inform your contacts with an auto responder?

How long do you intend to keep the gmail account?

chrissnell2• 5 years ago

I didn't do any auto-responders. Frankly, email is kind of dead for p2p comms with friends so there's not many people to inform. Most difficult part I've encountered was getting my dad, who is old, on board. He keeps emailing me at my GMail.

wly_cdgr• 5 years ago

All this kind of stuff just feels like cosplay unless you are gonna build out your own tier 1 (that also doesn't need other tier 1s)

Which isn't to say you can't learn a lot of useful practical stuff from it, just like you can from cosplay

jmartrican• 5 years ago

Once I "git reset --hard" the wrong repo and lost a lot of code. Had to use Dropbox's rollback feature to get it all back. Was not the first time Dropbox saved my butt. For me, a comparable alternative would need to have the same rollback features.

hyperion2010• 5 years ago

git reflog?

JeffDClark2• 5 years ago

I tried, and tried to use Spaces for image hosting related to a site that I am building but it never worked for me. DigitalOcean support was responsive but useless. I got the same basic response over and over again for days on end. I did finally get a response acking my problem and that it would be assigned to an Engineer. The ticket was then closed the next day with no further comment. I re-opened it and asked about the status of the engineer looking at the issue and received back the same response I had already heard nearly a dozen times. I then re-closed the ticket, deleted my Space, and went back to AWS. :(

I had heard good things about DigitalOcean, but I would not use them or recommend them to anyone after that experience. The one primary issue I had was a showstopper but there was a lot of other bugginess.

JoeCianflone• 5 years ago

That’s crazy. I’m using spaces on several projects and never had so much as a blip from them. If you don’t mind what was the showstopper issue for you?

JeffDClark2• 5 years ago

Erroneous 503s. I would get a 503 on every call. Uploading a single file with a single call would immediately 503. I never once got a successful upload. I simply switched the endpoint URL and credentials over to AWS S3 and it worked first time so it was not my code (though I suppose it could have been an issue in the library AWS's node S3 client). My theory was the request was somehow malformed based on what DO was expecting and it was returning a 503 rather than a 400. But the request did work with S3.

avsteele• 5 years ago

You can self host syncing service pretty easily:

  1) Get a Synology NAS. 
  2) Install the 'Synology Drive' package
  3) set up 'Quickconnect' (~DDNS)
  4) Remote backup can be though a number of providers (e.g. Backblaze) for cheap

No monthly fees. No domain needed. Storage is as cheap as the HDD's you stock it with. Your hardware and all your files are in your own house. Everything on BlackBlaze can be encrypted. You can set it up all through a webpage-based-GUI.

Technical, non-experts in computing stuff can do this. Not quite simple enough for the general public though.

https://www.synology.com/en-us/dsm/feature/drive

CharlesW• 5 years ago

QNAP's Qsync is their file sync service for QNAP NAS owners.

https://www.qnap.com/en/utilities/essentials

arcbyte• 5 years ago

This is what I use and I love it. Still working out what I want to do for backing it up to the cloud though. It has a very easy built on way to do that too, but im trying to figure out how "trust no one" I am with encryption and what exactly I really want to pay to keep around forever.

kwanbix3• 5 years ago

And what happens if somebody breaks in and steals your NAS?

timClicks• 5 years ago

All I need to do is write a Bash script that does the reverse and downloads from remote server to local folder. This could be another post.

A backup strategy without a recovery strategy already in place isn't a backup strategy.

jscholes• 5 years ago

To use this on Windows I suggest using Subsystem for Linux or Cygwin.

Unless you live inside these environments, an easier option would be rclone[1]. I use it on a regular basis to sync large loads to Digital Ocean Spaces, including its bandwidth limiting schedule to avoid saturating the residential uplink at inconvenient times.

[1] https://rclone.org

Merman_Mike• 5 years ago

including its bandwidth limiting schedule

Can you share more about this please?

The biggest (related) issue I run into running rclone from home is hitting the request speed limit on Digital Ocean Spaces. I frequently get the "Slow down!" error when backing up a bunch of small (<1mb) files.

arsome2• 5 years ago

I'd highly suggest Syncthing over this solution - no need to rely on 3rd parties for your data storage, let alone trust them with unencrypted copies of your data, if someone malicious worked at DigitalOcean and wanted to compromise you, it'd be trivial to put a malicious script in your Vault files. Especially if deplatforming is a concern something like Syncthing would avoid any reliance on a 3rd party that has visibility into your data.

anderspitman• 5 years ago

You may also be interested in rclone if you're not aware of it already.

vicken• 5 years ago

Take a look at duplicity. http://duplicity.nongnu.org/

It does encrypted, incremental backups and can sync to many protocols including S3, DO Spaces, Dropbox, rsync, Mega.co...the list goes on. Super easy to set up. This is my go to backup solution for the cloud.

uncledave• 5 years ago

Duplicity is crazy unreliable. Had so many problems with that from memory ballooning to corruption.

Only thing I’ve found which is even fit for purpose so far is rdiff-backup.

For personal stuff I’m just using iCloud and time machine to local disks.

saargrin• 5 years ago

this is absolutely not a dropbox replacement, this is a script that uploads your files to a cloud shared storage

dropbox client does so many more things (including compression,delta uploads,filehandle watch, incremental backup and so on)

i mean its fine if thats what you want,but that is not a backup solution by any means

NicoJuicy• 5 years ago

As a test case, i use box.com in combination with rclone and have daily backups for db and data of my servers.

It copies remotely to /{day of the month} + /{latest}

I could easily change my script to run every hour.

It works pretty nicely and i have a backup for 30 days

( Outside of the default backup daily ofc)

woudsma• 5 years ago

Funny, I've done the opposite in recent months.

Dropbox's killer feature is 'Online-only' storage IMHO, it allows me to store and access 300GB+ of music and other projects on my laptop without carrying an external HDD with me all the time. Every file is saved as a zero-byte file, which makes the files discoverable even though it's not stored on my disk. When I try to open the file it downloads it automatically, which works surprisingly well. And it's way faster than mounting DigitalOcean Spaces using s3fs for example.

Also Dropbox is cheaper: 12eu for 2TB, compared to DigitalOcean Spaces: 5eu for 250GB. And I can even access my files through the Dropbox API.

kwanbix3• 5 years ago

Love how well Dropbox works.

Sadly, for me, it is too expensive. It is 240 euros per year for a family of 6.

For comparison, the same plan in Google One costs 120 euros per year. Same for Office 365.

If they did 120 euros per year for the family pack I will migrate to them because I like Dropbox better at syncing.

kwanbix3• 5 years ago

(And Office365 actually gives you 1TB per user (so 6TB) and 60 mins of skype per user (so 360 mins), plus online, mobile, and desktop versions of office).

But somehow I have the feeling that Dropbox > Google Drive > OneDrive at syncing. Anybody can confirm?

EVa5I7bHFq9mnYK• 5 years ago

Why didn't he mention the cost? How does digital ocean compare to Hetzner's 1$/100GB/month snapshot offering?

dash2• 5 years ago

The article, and many of the posts here, seem like a baroque expansion of this (in)famous original response to the Dropbox launch: https://news.ycombinator.com/item?id=8863. I can get off Dropbox by... writing my own shell scripts?

What's sad is that increasingly people are feeling that the price for these easy, convenient services is too high, in terms of privacy loss, data lock-in, and so on. Are we going back to the early 00s, when the tech-savvy people would roll their own solutions and everyone else... just has to lump it? Are hackers just being too idealistic? Or do organizations like Signal show a possible way forward?

caymanjim• 5 years ago

I'm a long-time DigitalOcean user and I'm happy with most of their products, but I use Backblaze for S3-compatible storage. It's half the price of Spaces, and I get better transfer rates from my DigitalOcean droplet to B2 than I get to Spaces, even within the same DigitalOcean datacenter.

There are tons of companies offering S3-compatible storage, and there's not much difference between them, other than price. Performance capabilities are similar; they all limit bandwidth. All the ones I've evaluated charge for egress over a certain threshold. Some are more reliable than others. For personal use, there's not a whole lot of difference between them for me.

One really cheap alternative for some use cases is Office 365. For $99/year, you can get a family subscription, which allows six accounts. Each gets 1TB of S3-compatible OneDrive storage. If you can handle it being broken up into six 1TB chunks, it's by far the cheapest storage out there. For comparison, Backblaze is $5/TB/mo, and Office 365 gets you $1.37/TB/mo. I didn't end up using this option because the authentication mechanism was inconvenient for my automated use case and I didn't end up needing so much storage, but it's worth investigating if you're on a budget and don't need automated access via e.g. rclone mounts (or want to take the time to overcome the authentication issues I ran into).

I evaluated about a dozen S3 storage options for both backups and as backing storage for a personal Plex server (of media I own) and eventually settled on Backblaze because it's cheap, easy to use, and I didn't need more than 1-2TB in the end (since not a single one can keep up with streaming video bandwidth needs, I ditched the Plex idea).

dikaio• 5 years ago

I stopped using Dropbox as soon as C. Rice was put on the board. Tried tarsnap which was great did everything I wanted but I ended up using Arq instead, have had zero issues with it.

https://www.arqbackup.com/

Replacing Dropbox in favor of DigitalOcean spaces

Related Stories