Cloudflare R2 IA storage tier

https://blog.cloudflare.com/r2-events-gcs-migration-infrequent-access

by aofeisheng • 2 years ago

159 95 2 years ago

blog.cloudflare.com

nitsky• 2 years ago

Is the “data processing fee” any different from an egress fee in practice? Seems a little deceptive.

dannyw• 2 years ago

Yes. You can process it once to the standard tier, and egress as much as you want for free.

The differences stack up for say, a 1GB video that becomes viral and triggers terabytes in egress. You pay for 1GB, not terabytes.

It’s also an optional tier.

benterix• 2 years ago

The differences stack up for say, a 1GB video that becomes viral and triggers terabytes in egress. You pay for 1GB, not terabytes.

Under the condition that you actively monitor the usage and manage to "process it once" on time (and then "process it back"). Because otherwise you pay for terabytes - not in egress fees, but in processing fees. Or am I missing something?

ericpauley• 2 years ago

Yes, because in a well-designed setup files that are frequently accessed would be restored to standard tier. Ideally you'd only pay the data processing fee once when files transition from infrequently accessed to frequently accessed. There's a breakeven point at a data access rate of once every two months.

CharlesW• 2 years ago

Maybe the cold-to-hot migration "tax" is partially to prevent abuse?

"Data retrieval is charged per GB when data in the Infrequent Access storage class is retrieved and is what allows us to provide storage at a lower price. It reflects the additional computational resources required to fetch data from underlying storage optimized for less frequent access."

I like the "automatic storage classes" idea as well.

"…you can define an object lifecycle policy to move data to Infrequent Access after a period of time goes by and you no longer need to access your data as often. In the future, we plan to automatically optimize storage classes for data so you can avoid manually creating rules and better adapt to changing data access patterns."

dmw_ng2• 2 years ago

AWS already give you intelligent tiering for this, it's a very nice product but it's also just a nice way of hiding the same fees. Your $0.004/GB becomes $0.023/GB on first read for 1 month then $0.0125/GB for 2 months, so the average cost of storing it over those 3 months becomes $0.016/GB, and that's before considering monitoring fees

dmw_ng2• 2 years ago

At least magnetic disks are iops constrained, lower iops loads conceivably allow higher density, or packing different load patterns to the same devices. Say a 8 TB / 100 iops disk reserves 90 iops for a 1 TB a database service, that's 87% of the disk's capacity sitting free but only 10 iops to serve it with. Adding what is effectively an iops tax to discourage frequent reads is one way to make a mixture like this work (or another way to think of it - subtracting an iops discount)

Obviously example above is contrived, but same principle applies to a pool of 1000 disks as it would 1. You also don't escape this issue with regular hot storage either, there is still a (((iops * replication count) / average traffic) / max latency) type problem lurking, which would still necessitate either limiting density or increasing redundancy according to expected IO rate. This is one reason why some S3 alternatives with weaker latency bounds (not naming names, they're great but it's just not the same service) can often be made substantially cheaper, and why at least one of S3's storage classes may be implemented entirely as an accounting trick with no data movement or hardware changes at all

rc_mob2• 2 years ago

been waiting for that event trigering for a loooong time. I'll give it a go

aftbit• 2 years ago

How is data stored in this tier? Is it just on big slow SMR disks?

tills132• 2 years ago

Anecdotally, we have found R2 to be nearly half as slow to respond as the same request to S3 _proxied through Cloudflare_.

So... something isn't right here. Maybe a mechanical turk where a live human is fetching the object using Windows Explorer behind the scenes?

djhn• 2 years ago

Any chance you could convert ‘nearly half as slow’ to a percentage of the original response time? This reads like a reading comprehension puzzle and I don’t even know if my hunch is correct.

SahAssar• 2 years ago

Did you mean "half as fast"? The way you worded it sounds like you mean to say that s3 is faster, but it says that r2 is twice as fast as s3.

Lucasoato• 2 years ago

Does anybody know which consistency model Cloudflare offers compared to AWS S3?

mattdeboard• 2 years ago

Here are Cloudflare's docs on R2 consistency model https://developers.cloudflare.com/r2/reference/consistency/

gavinsyancey• 2 years ago

Backblaze B2 is cheaper than Cloudflare R2 IA. Hmmm

iscoelho• 2 years ago

This is expected. Like cloud providers, Cloudflare is intentionally not aggressive on pricing as it is a race to the bottom.

There are other ways to compete.

Guzba6• 2 years ago

B2 can be great but it is missing a lot of features when compared to other object stores so it isn't a good solution for every scenario.

As an example I investigated, to put a custom domain in front of a B2 bucket they suggest using Cloudflare and CNAME-ing a bucket subdomain (eg f000.backblazeb2.com) https://www.backblaze.com/docs/cloud-storage-deliver-public-...

Well if f000.backblazeb2.com is used for any other people's buckets too, which appears to be the case, I guess I am now able to serve other people's files from my domain? This seems terrible.

jdmarble• 2 years ago

I'm not sure I understand all of the nuances here (I'm no webmaster), but this is covered in the documentation you linked:

You must configure page rules to allow Cloudflare to fetch only your Backblaze B2 bucket from your domain. ... Otherwise, someone could use your domain to fetch content from another customer's public bucket. To ensure this does not happen, Cloudflare lets you use page rules to scope requests to your bucket.

nchmy• 2 years ago

and iDrive e2 is even cheaper. https://www.idrive.com/s3-storage-e2/

jjtheblunt5• 2 years ago

Isn't iDrive a BMW trademark for decades?

https://en.wikipedia.org/wiki/BMW_iDrive

_cenw3• 2 years ago

iDrive is slow and charges extremely high fees for usage above provisioned. They also have a history of increasing prices for individual contracts only a few weeks before renewal, so you can't possibly have enough time to move your data.

Hetzner Storage Boxes (2.50-3 EUR per TB) is probably the sweet spot. B2 if you need an object storage API.

ashconnor• 2 years ago

Rounded to the nearest TB rather than metered.

ac29• 2 years ago

Backblaze is unprofitable and publicly traded, a combination which cant last forever. They raised B2 prices 20% last year, I wouldnt be surprised to see more increases if they continue to burn through cash.

internetter2• 2 years ago

Backblaze is unprofitable and publicly traded

So is Cloudflare

kjkjadksj4• 2 years ago

I can’t get over the fact how storage is still so expensive in 2024. Lowest you can get is probably $5 per tb a month from any of these companies. A new tb hdd is probably $25 today. Where does the money go, into c suite car payments?

fire_lake• 2 years ago

It’s availability that’s expensive.

Hamuko• 2 years ago

All those nines cost extra.

godelski3• 2 years ago

I really believe there's a missing market. I think there would be big business in building servers for homes. Where you prioritize low power usage and low noise, and do not need blazing speeds or frequent/high access. Cloud is great, but some things should still be stored at home. Home NAS systems are a bit odd and difficult to expand (nowhere near what a rack is).

My argument would be that this would be helpful with the high adaptation of things like Ring doorbells and other camera systems at home. Where people can store their own data and provides better security & privacy given you need not rely on a data connection to store that footage. It also would be extremely worthwhile if we are to see personalized LLMs become common and tools like home assistant. You wouldn't want that running off-site. In fact, I'd rather call home from my mobile LLM than call FAANG (or anyone else with teeth).

I just think buying used servers on ebay or trying to throw together a home rig is harder than it needs to be. I'm confident the demand exists but it is unfortunately a field of dreams scenario. Many people will not know they want it until it exists (I can say my parents would love this but they don't understand the first thing about technology so all they can do is complain about Google/Apple having all their data rather than express how they want to store their own).

aseipp• 2 years ago

Well, spinning rust HDDs stuck in your server have no actual parallelism and aren't highly available, replicated, etc.

How much do you think 1TB of storage should cost?

sophacles2• 2 years ago

You're comparing apples and oranges here....

A $25/TB drive is not the only expense that $5 goes towards:

* there's actually probably 2 or more HDs holding that TB, since the business is promising that the data won't be lost

* theres the computer(s) that hold that HD.

* theres the electicity, bandwidth and space rental costs for those computers

* theres the cost of employees to make sure that the computers keep running.

* theres the cost of the marketing so that you know that the service is available

* theres all the book-keeping, taxes, cc fees, etc that need to be paid on the recurring charge

* there's (hopefully) profit for the investors/owners

and so on.

Also, on your side you should consider several of those factors yourself to do the comparrison:

* how much do you consider the time spent managing your hdds to be worth? (if you're a business this is employee-hrs, if you're talking about for yourself privately, there's still a value you should attach to your own time)

* do you have backups? If so, what does it cost to put them offsite? (In terms of space rental or favors traded, and your time)

* electricity, etc

* how much is it going to cost you to learn to reliably store your data (in terms of up-front cost, time spent, etc)

* and of course hard drive costs

RantyDave• 2 years ago

Backblaze have written many blog posts on how they go from a few thousand hard drives to a business. For me, the most interesting part was they went from six generations of self designed storage pods to "fuck it, just buy Dell". Long story short: these businesses are surprisingly complicated.

https://www.backblaze.com/blog/next-backblaze-storage-pod/

smileybarry• 2 years ago

IIRC Backblaze B2 charges for egress, while CloudFlare does not.

vineyardmike2• 2 years ago

They do not charge for egress either.

https://www.backblaze.com/cloud-storage/landing/ad/use-cases...

historynops• 2 years ago

A fairly unrelated point, but its so strange how companies that underpin a lot of the internet struggle in the stock market. While we all wish we had sold our tech stocks in 2021, Cloudflare still hasn't recovered.

Guzba6• 2 years ago

I believe Cloudflare (and many other cos like it) have never produced operating income. They are growing and obviously important and potentially very profitable in the future, but when discount rates are much higher and you add in some uncertainty, one could argue they don't look as hot as they used to.

simfree• 2 years ago

Cloudflare has a very dysfunctional sales pipeline. Their free, premium and self-serve offerings might underpin the internet, but the highly profitable offerings that are gated behind their sales teams are not getting sold. Too many of the clients that they should be selling to.

Magic Transit (bring your own ASN), classic website DDoS protection (above the Business $200 tier, which has low, undisclosed data limits in regions like New Zealand) and ilk all require interacting with the sales rep, and unless your paying 5 figures a month they are disinterested.

There is a whole market out there between $300 to $2000 a month that Cloudflare could tap without making new infrastructure but is actively being ignored.

alphabettsy• 2 years ago

This.

They lock a lot of features behind an Enterprise plan where they could allow them to be added to a lower plan.

In general, I just hate working with sales reps and would rather avoid a company altogether if I can’t sign up without talking to them.

lijok• 2 years ago

Hit the nail on the head.

Wanted to byt their SASE DLP & Remote Browser Isolation as a startup. Sales wouldn't even talk to us

teruakohatu• 2 years ago

undisclosed data limits in regions like New Zealand

Can you please explain what this means?

no_wizard2• 2 years ago

Not to mention they have on multiple occasions made significant internal changes (including layoffs) to their sales organization. I have a feeling if the public were to get an introspection into their sales pipeline it would be eye opening, and not in a good way

kjkjadksj4• 2 years ago

It is bizzare. All the old guard foundations of society type companies that the world relies on for modernity have stocks that barely budge but pay out decent dividends. Maybe tech stocks that have grown to such a position should consider paying out dividends instead of failing to chase exponential stock price growth while still clearly doing a lot of productive things. I expect the shareholder boards prefer the chance of exponential wealth over steady returns and prevent this mindset from emerging.

tschellenbach• 2 years ago

does anyone know which products of cloudflare have the most revenue?

rc_mob2• 2 years ago

probably the CFO of Cloudflare inc knows

ricopags• 2 years ago

We're onboarding to Cloudflare MagicWan and want to use them for logging, which they do to 's3-compliant' buckets.... on Google or Amazon.

I was pretty surprised at the lack of dogfooding, wondered if it's an oversight, on somebody's Gantt, or just not something R2 can handle for some reason.

viraptor2• 2 years ago

Yeah, the integration and production readiness of their non-core offerings is not perfect. I'm dealing with R2 and another service and you can tell they fell more like... specifically integrated features, rather than fully modular services you can choose to use as you want. Like the workers have possible R2 bindings, but you can't use those in a fetch() call - you have to use S3 compatible endpoint instead.

AWS has its own issues, but the push to have everything talking over API did wonders for the ability to use them as you want.

jakubadamw• 2 years ago

Like the workers have possible R2 bindings, but you can't use those in a fetch() call - you have to use S3 compatible endpoint instead.

Sorry, could you please elaborate? Why can you not use a binding to an R2 bucket – and perform operations on its objects – in a `fetch()` handler of a worker? Or did I misunderstand this statement?

thrixton• 2 years ago

So pricing is 1c / GB-month, compared to S3 IA at 1.25c / GB-month, a decent saving but not massive, no archive or deep archive options though, I wonder if / when these will come.

What sort of negotiated rates can you get from AWS for bandwidth I wonder, at the moment, that’s seems like the only real benefit from CF I think.

Cloudflare R2 IA storage tier

Related Stories