Curl native support for –aws-sigv4 (2023)

https://how.wtf/aws-sigv4-requests-with-curl.html

by paulddraper • 2 years ago

20 17 2 years ago

how.wtf

paulddraper• 2 years ago

This is helpful for calling AWS Gateway API with AWS_IAM authorization.

Or for calling AWS APIs in containers, etc. without installing heavyweight AWS CLI.

Galanwe• 2 years ago

Happy to see that, though it would have been nice if it retrieved the credentials automatically instead of explicitly requiring them.

lopkeny12ko2• 2 years ago

Why are we celebrating proprietary, vendor-specific-extensions in our open source tools that are, by nature, supposed to operate in a vendor-agnostic manner? This flag makes zero sense for anyone who isn't an AWS customer. This is even more insulting considering that "being an AWS customer" is not free (both as in beer and freedom). It really surprises me that this made it into curl.

I've always been a fan of Terraform's provider architecture--all the vendor-specific bits are implemented as third-party plugins that implement a standard API and communicate with Terraform itself in a client-server model.

aeyes• 2 years ago

Is it really proprietary? We now have a lot of S3 compatible servers, MinIO for example requires this type of signature for requests.

alexhornbake• 2 years ago

^ this.

The AWS v4 signing method is a spec for how to do HMAC request signing. It is not limited to use only with AWS services.

You can use it to sign requests on any service, and there are many implementations. Here is an example of one I’ve used in the past: https://github.com/auroratechnologies/vangoh

Or as mentioned above: https://github.com/minio/minio/blob/master/cmd/signature-v4....

gsk22• 2 years ago

Isn't that the beauty of open source? That we are empowered to make tools that fit our needs and the needs of our users?

I don't see why open source tools should be intentionally crippled in the name of some sort of "purity". Implementing this feature costs nothing (it is an open standard, unencumbered by licenses or patents), and you can continue to use the tool as if the vendor-specific option doesn't exist.

yjftsjthsd-h• 2 years ago

It's less about purity and more about overhead. Adding this feature adds friction to refactoring the code, adds surface area for vulnerabilities (and the user ignoring it won't make that go away), and makes the binary bigger. The cost/benefit ratio might well be favorable, but implementing this feature absolutely costs something.

akira2501• 2 years ago

celebrating proprietary, vendor-specific-extensions in our open source tools

It's not proprietary nor vendor specific. I'm not sure what makes cURL "our" tool?

by nature, supposed to operate in a vendor-agnostic manner?

Does the inclusion of this feature mean a different feature cannot coexist with it? Was it so absurdly complicated to develop that it took time away from other integrations?

This flag makes zero sense for anyone who isn't an AWS customer.

Do you also dislike QUIC or RTMP? How about NTLM?

that "being an AWS customer" is not free

Yea but "using AWS protocols outside of AWS" _is_ free.

It really surprises me that this made it into curl.

They should turn down useful but impure contributions? The lead developer of cURL still has a day job.

kelnos• 2 years ago

Because some people believe tools exist to make lives easier, not to evangelize some sort of ideology. A lot of people use AWS; adding helpers for it into commonly-used tools can make people's lives easier.

And beyond that, there a bunch of clones of various AWS APIs, both in other proprietary products, and in open source alternatives. I bet this functionality works with those, too.

Scaevolus2• 2 years ago

It's quite a clean extension that can be disabled at compile time. It doesn't complicate the rest of curl, and provides a useful way to invoke any AWS API from the CLI without needing a giant AWS library dependency.

It's well worth a thousand lines of code.

https://github.com/curl/curl/blob/b9fea261e632b9493ea6038bfb...

akdev1l• 2 years ago

As some other people pointed out. AWS SigV4 is just a spec on how to apply HMAC to validate/authenticate requests.

You can implement quite easily in any language without touching any Amazon tooling or infrastructure.

thomastaylor• 2 years ago

I appreciate the shout! To accommodate the influx of people coming to the blog, I updated the post to include IAM role assumption credentials as well. I hope you all enjoy!

NanoWar• 2 years ago

I had fun recently to make such a request in C#. In the end I ported the code from the aws typescript implementation to C# which was surprisingly easy: https://gist.github.com/NanoWar/788860d74c49f337f4c3dab6dd24...

dmart• 2 years ago

Just mentioning that I love AWS' signed request model. It's so much easier than dealing with OAuth for APIs and I wish it would catch on more broadly. It's inspired an IETF RFC (https://datatracker.ietf.org/doc/html/rfc9421), so we'll see.

Curl native support for –aws-sigv4 (2023)

Related Stories