A New Hope for Object Storage: R2 Enters Open Beta

https://blog.cloudflare.com/r2-open-beta/

by mkevanz • 4 years ago

68 38 4 years ago

blog.cloudflare.com

JonoBB3• 4 years ago

We've been waiting for this since it was announced. We have > 2PB of media heavy data, and transfer a lot via S3 to the internet.

This will reduce our monthly S3 charge from tens-of-thousands USD per month, to ~$200.

greg-m2• 4 years ago

This is what we live for :)

ddorian433• 4 years ago

Please post a reply here when you get the "you're using too much bandwidth" email and the price that they'll quote you.

Wallacy• 4 years ago

Thats problem is for CDN product. On R2 like other Developer products (KV, Worker, DO) you are already paying the bandwidth over the storage price. All those products can be used to HTML and non-HTML product.

https://www.cloudflare.com/supplemental-terms/#cloudflare-de...

Thats the major problem of the CDN product, they dont say that are no traffic fee, they just say they will not charge you "except" on some cases (that you never know). But other products they are very specific of the egress fee, like Stream, Magic Transit, Spectrum, etc.... They should fix that matter on CND, no one like that.

Sure, i dont know what will happen if someone just try to make a youtube like over R2... They probable "want" that....

getcrunk• 4 years ago

The storage cost on r2 is .015/GB. Might want to redo the math

JonoBB3• 4 years ago

I should have specified "This will reduce our monthly S3 _transfer_ charge from..."

vlovich12310• 4 years ago

I’m an engineer on the R2 team. Happy to answer any questions in this thread.

JonoBB3• 4 years ago

If I remember correctly, there was some discussion around R2 acting as a proxy in front of s3. So, if R2 does not yet have the asset, you would automatically pull it in from S3, and from then on R2 would serve it directly.

Is that still on the cards?

ignoramous3• 4 years ago

Not a Cloudflare employee, but: Yep, Cache Reserve is in private beta https://blog.cloudflare.com/introducing-cache-reserve/

greg-m2• 4 years ago

Yes, still in the cards. Hoping to release this before GA - it'll be a part of our migrator.

brenogazzola• 4 years ago

Any advice/guides on how to get this working with Rails ActiveStorage?

vlovich12310• 4 years ago

Should be the same as any other S3 backend. We don’t have examples for ActiveStorage per se but we have boto3, AWS JS SDK, and rclone [1].

[1] https://developers.cloudflare.com/r2/platform/s3-compatibili...

ignoramous3• 4 years ago

1. TFA goes R2 clients will be able to enable caching (in the near future). A couple questions:

1a. Cloudflare Cache is free for users (up to 500MB per file for unlimited files), but if R2 uses it transparently, are reads from cache metered per R2 pricing? And what about the file-size limits that apply to free/pro accounts?

1b. How is strong-consistency guaranteed despite caches in front of the bucket? I guess cache-invalidation can be done through Durable Object (DO) Alarms or some such? Or, would this guarantee no longer hold?

2. TFA mentions in the passing that R2 utilizes DO (for presumably metadata / journal?). Cloudflare once wrote in detail about how DO is used in a pretty novel way to power wrangler tail, and so I was wondering if there's something novel about R2's use of DO, too?

3. In-Worker API: Are customers metered for both In-Worker API (duration and invocation) and R2 (Class A / Class B)? If so, that makes Class B operations almost twice as expensive than S3 (excluding egress).

4. Any ETA on "Public buckets"?

5. Has Cloudflare rolled out newer server designs just for R2 (aka Dropbox's MagicPocket)?

6. Is serving media content (say, audio and video) an acceptable use-case for R2?

7. Any plans for Kinesis Firehose (a managed-ingestion pipeline) esque product to front R2?

Thanks.

vlovich12310• 4 years ago

1a. I don't think we have anything to share with regards to the logistics of putting cache in front of R2 or how that product will work.

1b. Unavoidably as you guess you'd be giving up the consistency. Presumably the consistency guarantees would be part of the cache-control header you associate with the object. Cache purging would probably work similar & we have exciting upcoming news about major performance improvements for that. But again, since we don't have any idea of what that product looks like, hard to say.

2. There is. I have a bunch of technical blog posts written but we decided to punt on providing that detail for a few months just to give everyone some breathing room. R2 does have one important extension we got the DO team to build just for us (not 1 instance of the code instantiated). That piece will only be available for R2. R2 is also driving performance improvements & API improvements to DO that will almost certainly percolate out to external users.

3. The equivalent model would be AWS Lambda + S3 so I think we're significantly cheaper (+ Workers is faster than AWS lambda - R2 is still in open beta so not useful to compare perf against S3 yet). Yes, if you're using it to emulate public buckets, then it'll be a bit expensive (depending on egress charges you might otherwise face)

4. Soon. It's very high on our list alongside pre-signed URLs

5. Nothing to share at this time.

6. Yes. From Matthew Prince aka eastdakota [1]

7. Nothing to share at this time.

[1] https://news.ycombinator.com/item?id=28682885

ehutch79• 4 years ago

Does the worker api support multipart uploads?

taylor123452• 4 years ago

Not yet. The current in-worker api is described at https://github.com/cloudflare/workers-types/blob/master/inde...

ianopolous2• 4 years ago

Thanks for the awesome product! I have a few questions:

1. Are the rate limits for all users on a given bucket, or by source IP as some other providers do?

2. What is the plan for the rate limits post beta?

3. Do you support the x-amz-content-sha256 header. i.e. If a put contains that header and the content hash doesn't match is it rejected?

4. What does it mean to support pre-signed URLs? Can't a client pre-sign a URL that anyone else can use? My understanding is they are the same as normal usage from the server's perspective, just the key holder signs the request, and the other person actually makes the request.

5. Why is the read limit smaller than the write limit?

taylor123452• 4 years ago

I'm also an engineer on R2 team.

1 & 2 - We're reserving some ability to rate-limit during beta as we figure out how best to scale our systems under load. I'll defer to vlovich & greg-m for post-beta plans.

3. We do support that header. Mismatching content hashes will be rejected during write. If you ever experience otherwise, please report it.

4. We only support sigv4 auth via header currently: https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-auth-u...

With presigned urls, we will support sigv4 in url parameters https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-query-...

5. See #1. We'll be improving the throughput of both operations, but currently reads have higher throughput than writes.

ignoramous3• 4 years ago

Neat!

   cl = boto3.client(
      's3',
      aws_access_key_id=access_key_id,
      aws_secret_access_key=secret_access_key,
      endpoint_url=endpoint,
      config=Config(
          region_name = endpoints[endpoint_name].get('region', 'auto'),
          s3={'addressing_style': 'path'},
          retries=dict( max_attempts=0 ),
      ),
  )

dict "endpoints" is undefined in the example code above, replace that line with:

  region_name = 'auto',

----

While,

   an S3-compatible API, which exposes your bucket on a URL of the form bucket.account.r2storage.com.

Should be?

   an S3-compatible API, which exposes your bucket on a URL of the form account.r2.cloudflarestorage.com/bucket

vlovich12310• 4 years ago

Yup. Sorry about that. I have a complicated Boto3 script I reduced but didn’t test and typo on the URL name.

We’re fixing the blog post.

captn3m0• 4 years ago

Amused by the Star Wars reference in the title. (D1 should have been called D2, just to get the complete name).

vayne• 4 years ago

Yeah, but that could be another sequel. D1 is one minus D2

vlovich12310• 4 years ago

https://twitter.com/eastdakota/status/1524392750342238214?cx...

eins12342• 4 years ago

Got any examples for https://github.com/aws/aws-sdk-js-v3 or https://github.com/mhart/aws4fetch?

I haven't been able to get either working:

  const s3Client = new S3Client({
    endpoint: 'https://my-account-id.r2.cloudflarestorage.com',
    credentials: secrets.r2Credentials,
  })

  await s3Client.send(
    new PutObjectCommand({
      Bucket: 'my-bucket,
      Key: 'test',
      Body: JSON.stringify({hello: 'world'}),
    }),
  )

^ This fails with a 500

  const r2Client = new AwsClient({
    ...secrets.r2Credentials,
  })

  fetch(
    await r2Client.sign(
      'https://my-account-id.r2.cloudflarestorage.com/' +
        '?Key=test&Bucket=my-bucket',
      {
        method: 'put',
        body: JSON.stringify({ hello: 'world' }),
      },
    ),
  ).then((response) => console.log(response.status))

^ This fails with 401

I've verified that the key has full access and even rerolled it to make sure I didn't mess up the copy paste, but still no luck.

Any ideas where I could be going wrong?

vlovich12310• 4 years ago

We have several examples [1].

I think you need to set s3ForcePathStyle to true for the first example (or however you do it in the v3 version of the SDK[2]). For the latter, we don't support pre-signed requests. Feel free to ask on discord [3] too. Might get more timely responses.

[1] https://developers.cloudflare.com/r2/examples

[2] https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clien...

[3] https://discord.gg/cloudflaredev

eins12342• 4 years ago

Thanks for the resources! I managed to get both working in the end. Some notes for future readers:

For SDK v3, I just needed to add the forcePathStyle flag.

For aws4fetch (note that this isn't a presigned request, it's just signing the request for Sig V4 like the official SDK would), I needed to change the fetch url to https://my-account-id.r2.cloudflarestorage.com/my-bucket/key.

fjfbsufhdvfy3• 4 years ago

What stops an attacker from generating millions or billions of requests to your bucket, generating unaffordable bills from the class B charges?

When using the "free" cache product, solving this is Cloudflare's problem - they have a lot of incentives to stop people from being able to do that as it would otherwise cost them money.

However, when using the R2 product, Cloudflare sure would be incentivized to do whatever they can to make all these requests succeed so they can bill you for them.

Before someone tries bringing up their rate limiting product, that is more than an order of magnitude more expensive than R2.

vlovich12310• 4 years ago

I think what you’re looking for is Bot fight mode, which is a product we have that lets you block bots and customize how you react to them. There’s a free tier and capabilities get more powerful as you price things out [1]. I would imagine that when we add support for public buckets, we’ll be thinking about how that integrates with Bot fight mode (e.g. enabling the free tier automatically).

Putting cache in front of public buckets is also something you’ll be able to do.

Rate limiting is very similar to spending limits and, at the scale you’re talking about, they’re just expressing the same concept from different perspectives. R2 is currently naturally rate limited anyway because we can’t go faster but that will obviously evolve as we improve the system. Additionally, you can’t currently have public buckets (at least not without exposing it through a Worker you write). The main use for rate limiting and spending limits is more about being OK choosing to have an outage because of a spike in load (e.g. your project going viral unexpectedly on HN before you’ve cost-optimized it for that kind of scale).

From a motivation/incentives perspective, Cloudflare is a SaaS company focusing on security and trust. We’re never incentivized to have unhappy customers, no matter how big of a bill we can deliver for any given month. Doubly so when that comes because you’re under attack vs you’re having success.

So TLDR:

1. Bot fight mode is probably useful as a DOS mitigation tool when hosting a public bucket.

2. When we add virtual pathing, buckets are hosted under your account domain. Unlike with other providers, people can’t scrape public records to find what buckets you’ve exposed (intentionally or otherwise) to the public.

3. The product is currently naturally rate limited.

4. Rate limiting isn’t a viable solution here because an outage is not meaningfully better from us charging you insane amounts of month for that month. If you get an unexpectedly large bill, please reach out to Cloudflare support.

5. Public buckets don’t actually exist yet.

6. Cloudflare is in the business of security and trust and we’re incentivized to do our best to engender that trust.

Does that speak to your concerns?

fjfbsufhdvfy3• 4 years ago

Sounds good! Yeah, my concerns are more related to public buckets, I suppose, and those aren't out.

I have some more thoughts on those:

- Will adding the cache in front of a public bucket come with the necessary ToS updates to allow serving the media content in the bucket?

- It would be nice if there could be some kind of timed authorization for downloads (like a signed token in a header or something?) that does not count towards class B charges if it fails. Currently you can implement this with a worker, but it actually still causes a charge to run the worker, so is mostly ineffective.

A New Hope for Object Storage: R2 Enters Open Beta

Related Stories