Introducing Object Storage

https://www.digitalocean.com/products/storage/object-storage/

by rsivapr • 9 years ago

98 41 9 years ago

setq3• 9 years ago

If there's anything I've learned about object storage over the last year, it is: don't use object storage unless you need it for a particularly narrow use case of silly data volumes or CDN storage for a massively distributed content network. Anything else, forget it.

It rarely if ever works properly with standard Linux or Windows tools (s3), it has a rat's nest of arbitrary restrictions which require a language lawyer to decypher (s3/iam/vpc/roles), the APIs are vendor specific and sometimes even region specific (s3), the APIs are obtuse (s3 multipart), the clients are buggy (boto/boto3), suddenly you inherit extra costs and configuration requirements if you want to do something like expose it over http (route53/cloudfront/s3), credential storage is a nightmare for distribution compared to rsync/ssh etc. Ugh.

Please note I have used Google Storage as well and all of the above also apply.

The only thing that is positive is capital expenditure is low.

chris_wot• 9 years ago

Is there a stand for object storage?

setq3• 9 years ago

openstack swift supposedly but this is a slightly incompatible S3 implementation that nothing else talks.

The filesystem works pretty well though ;)

indogooner• 9 years ago

I think AWS CLI is pretty handy for lot of tasks. Not sure if you are referring to which multipart API of s3[1]. There is a higher level API for quite some time now which is very convenient. Agree about documentation for iam/vpc/roles being arcane but tbh I have no idea how that could be simplified.

[1] http://docs.aws.amazon.com/AmazonS3/latest/dev/usingHLmpuJav...

Edit: Added link to high level API

CobrastanJorji2• 9 years ago

credential storage is a nightmare for distribution compared to rsync/ssh.

I don't understand this argument. Why is it harder to store credentials for S3 or GCS than it is to store credentials for rsync or ssh?

setq3• 9 years ago

Because everything has no ownership, no standard place to put things and no logical consistency.

zedpm• 9 years ago

Slowly, very slowly, DO is creeping towards having enough features to peel users off of AWS and GCP. Block storage, load balancers, and now blob storage...if you don't need managed queues or databases, you can probably save some serious coin by running services on DO.

S3 sets a high bar in terms of durability and availability, so it will be interesting to see how well DO can compete, and it will take a long time to gain the same level of trust that S3 has earned.

SamLevin88• 9 years ago

Good to know. Am in the throes of choosing between both for a simple project

raiyu9• 9 years ago

VPC is in the works at the moment, still a bit too early for a release date, but the team has been on it for about the past 3 quarters.

Arcsech• 9 years ago

I wouldn't be surprised to see DO either roll out DaaS next or partner with someone (Compose.io?) to do it. That's one of two things (the other being VPC, which I know they're working on per HN posts elsewhere) that make AWS/GCP tempting over DO for me.

That said, I kind of hope it's not Compose.io - they seem ridiculously overpriced for PostgreSQL compared to AWS's and GCP's offerings.

lftl• 9 years ago

IBM owns Compose.io now, so I'd be a little surprised to see them launch anything on DO.

tedmiston3• 9 years ago

VPC on DO is the #1 feature request that stops my team from considering it.

I use DO personally for small things and like it a lot, but VPC is a hard requirement for company work.

gbrits• 9 years ago

Using managed queues (SQS) from AWS. The rest is on Vultr. That mix-and-match works perfectly for my use-case. In other words, no need to be able to run everything on a single platform to get cost gains.

ocean11• 9 years ago

There's pretty much zero information on that page, so why should I care?

raiyu9• 9 years ago

We roll out a public beta test with existing customers to ensure performance and reliability before doing a GA release. This allows customers to use the new product and provide feedback as well as help us to ensure a great level of service when the product goes full GA. So it's really a way for customers to test out new products early, but the specific details are specifically light so that if there are any major changes in the beta period we can roll them out.

fredsted• 9 years ago

I think it's a little weird to say "Introducing Object Storage" when you're not introducing anything at all. There's zero information about it!

Is it S3 compatible? Does it compete with S3? Does it supports static file hosting? How do permissions work?

I think it would make more sense to communicate that you're looking for beta testers for an object storage service.

ctrlrsf• 9 years ago

Anybody have or see pricing after early access period?

raiyu9• 9 years ago

We aren't providing pricing information till full GA which is later in the year. All beta users will be given 1TB of free object storage during the beta period.

justrobert2• 9 years ago

I think the lack of a pricing model and how short the free period is (until Oct 2017) makes it hard for many users to even justify testing it.

For me it seems if a developer has to spend more than an hour testing this new product they are better off using S3.

pramodliv1• 9 years ago

I wonder what technology they use under the hood. Ceph? GlusterFS? Custom software?

safeharbourio• 9 years ago

they chose to make their own stack afew years ago instead of going with the already available and business friendly openstack, going by the same reasoning and the long amount of time they took to reach here, it highly likely to be a custom thing.

pinewurst2• 9 years ago

If OpenStack is "business-friendly" then it's very particular about who its friends are.

CSDude• 9 years ago

I think Ceph because it was mentioned in their job openings

sandeep048• 9 years ago

Incase anyone missed the recent cloud firewall announcement https://blog.digitalocean.com/cloud-firewalls-secure-droplet...

This might be possible because of the advances in software defined networking. Cloud firewall removes the hassles of setting up complex iptables in the name of network security and frees those extra cpu cycles and memory utilization.

DigitalOcean is getting better at solving common VPS use cases. I like it :)

megamindbrian2• 9 years ago

Um. Ok, something is wrong with your UI. There is no options to use Object Storage after signing up and required to enter a credit card. The drop down menu font is blurry. Support page takes about 30 seconds to load.

raiyu9• 9 years ago

This is an invitation for beta access, you sign up to request access and then and invitation to be a part of the beta access will be emailed separately.

megamindbrian2• 9 years ago

I already cancelled my account.

acrosync• 9 years ago

I'm the developer of Duplicacy (https://github.com/gilbertchen/duplicacy), a new generation cloud backup tool based on the idea of Lock-Free Deduplication(https://github.com/gilbertchen/duplicacy/blob/master/DESIGN....). Currently Duplicacy supports many cloud storages (S3, Azure, GCP, B2, etc), and I would be very interested to build one for DO Object Storage.

Is it possible to get early developer access to the API?

Introducing Object Storage

Related Stories