Show HN: Prefixed, dual-token, base58 encoded API Keys

https://github.com/seamapi/prefixed-api-key

by seveibar • 4 years ago

100 54 4 years ago

seveibar11• 4 years ago

Author here: I was curious what HN would think of this token style, we're currently using it for getseam.com . I think it has a lot of advantages over other token styles (e.g. double-click to select, standard alphabet, compatible with secret scanning). Although this is a typescript library, we originally designed this API key for use in Ruby on Rails and the pattern should be fairly portable.

joshuawarner32• 4 years ago

On iPhone the long-press-to-select doesn’t cross the underscore boundary, so you end up just selecting part of the token. Maybe you can do without those underscores?

__sy__2• 4 years ago

**shakes fist at sky/apple**

koolba2• 4 years ago

Couple comments upon looking at the actual code:

- You can promisify randomBytes once and reuse it rather than twice for every invocation

- There shouldn’t be a default value for the company name or people will end up using it.

- The company name isn’t validated so it could contain underscores which would cause issues with the short token parsing as it assumes it’s the second “chunk”

- The equals comparison of the hashes for the secrets is not timing safe. It’s not as bad as if they were plain text but it does short circuit due to how string equals works. Use the actual built in timing safe equals on the Buffer hash (not the stringified hex).

seveibar11• 4 years ago

thanks for taking a look! I've created an issue on the repo and should be able to address these tonight :)

jffry2• 4 years ago

Thanks for sharing! As somebody who has been bitten before by copy-pasting a key I appreciate that as the headlining benefit :)

Thoughts from an ergonomic perspective:

- Base58 is nice and makes easy-to-handle values which is nice

- Naming of shortToken, longToken, and token are confusing to me, because it makes it sound like all of these values are tokens/secrets of some sort, rather than components of the token. Not to bikeshed, but what jumps to my mind is more like "prefix_id_secret", which helps make it clear what role each one plays.

- I don't think keyPrefix should have a default, I think your function should raise an error if it's not provided. Otherwise more than one user of your library is going to end up with "mycompany" keys in the wild.

- You probably ought to validate that keyPrefix does not contain an underscore and raise an exception if it does, otherwise that way lies pain for people trying to handle these keys.

Thoughts from a security perspective:

- You should probably use crypto.timingSafeEquals(buffer, buffer) [1] instead of comparing strings

- If you're trying to store the secret's hash, isn't something like bcrypt/scrypt preferred over raw sha256 these days?

[1] https://nodejs.org/api/crypto.html#cryptotimingsafeequala-b

koolba2• 4 years ago

If you're trying to store the secret's hash, isn't something like bcrypt/scrypt preferred over raw sha256 these days?

Not for this type of use case. If a secret is long and generated from a cryptographically secure source (eg /dev/urandom), then any cryptographic hash function is fine as brute forcing the secret itself is not feasible. A single pass of sha256 is fine and presumably you’d be doing many of these operations each second in a high throughput application.

“Slow” hash functions like bcrypt or scrypt are for user provided secrets that might not have a large amount of entropy. It’s fine for something like user authentication but would be way too slow and pointless for API keys.

seveibar11• 4 years ago

Appreciate the thoughts on the naming/exceptions and thanks for taking a look at the implementation! Definitely making some adjustments tonight.

I think your thought on bcrypt/scrypt vs SHA256 is super interesting here. The long token is treated a lot like a password, so we should treat it similarly and use slow hashing. However, unlike a password an API key is repeatedly used for authentication instead of being exchanged for a session token. I don't think this meaningfully changes anything- so I think you're right that bcrypt/scrypt would be a better choice!

Edit: Also see koomla's answer!

remram• 4 years ago

If you are prefixing it, why not prefix it with a full domain name? mycompany_com_BRTRKFsL_51FwqftsmMDHHbJAMEXXHCgG is not much longer than mycompany_BRTRKFsL_51FwqftsmMDHHbJAMEXXHCgG and allows reporting leaked secrets (for example using the GitHub scheme [1]) without having to look up the URL in some sort of registry, e.g. POST https://mycompany.com/.well-known/report-leaked-secrets

[1] https://docs.github.com/en/developers/overview/secret-scanni...

edit: See also the discussion from last time: https://news.ycombinator.com/item?id=28296864

seveibar11• 4 years ago

I think this is an excellent idea! However, we didn't like the aesthetic of the version that included the domain- I'll admit it's not great reasoning.

adastra222• 4 years ago

Don’t use base58. Even bitcoin, which originated or at least popularized base58 has moved away from it. They are a pain to encode and decode, with a naive implementation requiring a bignum library, and they don’t save you much in the end over a base32.

epse• 4 years ago

But you don't need to encode or decode anything here? Tokens are just random strings?

splix• 4 years ago

Can you tell more about the prefix? Not very clear to me. Is it issuer id or user id? And if it's later and is user provided, how it helps with scanning

bluehatbrit• 4 years ago

It's the issuers company or product name. GitHub for example prefix their tokens with "gh_" which makes it easy to scan for tokens uploaded to repos or even across the web if they wanted.

GitHub has secret scanning features built in now, with a prefix of your product name or company name you could easily create a regex of sorts to find when someone has uploaded an API key for your application to their GitHub repo and revoke the token or email them.

mdaniel• 4 years ago

I don't follow what the security property is of involving sha256 against the b58 encoded value? If it serves as a checksum against tpyoss, wouldn't one wish to include the shortToken also? Otherwise, it's just as much attacker controlled content as the b58 version is, as best I can tell

As an implementation observation, I am also on a life-long campaign to rid the world of `split(...)[-1]` type manipulations, because they lack the context of a more rigorous "parsing" style. In this specific case, it allows attackers to smuggle almost arbitrary characters between the shortToken and longToken:

    checkAPIKey("alpha_BRTRKFsL_and this one time at band camp\r\nContent-Length: 0\r\n_51FwqftsmMDHHbJAMEXXHCgG",
    "d70d981d87b449c107327c2a2afbf00d4b58070d6ba571aac35d7ea3e7c79f37")

Also for your consideration, the code currently has duplication:

    export const extractLongTokenHash = (token: string) =>
      hashLongToken(extractLongToken(token))
    // ...snip...
      longTokenHash: hashLongToken(extractLongToken(token)),
    // ...snip...
    ) => hashLongToken(extractLongToken(token)) === expectedLongTokenHash

and my experience is that it's so easy to remember to update one and forget to update the others

seveibar11• 4 years ago

Thanks, yes I agree the ".split(_)[-1]" is ugly and there should be validation on the key prior to operations on it!

Making an issue :)

benatkin7• 4 years ago

I like it. I'm not sure that "short token" is the right term for that - perhaps "token id". Also the only occurrence of "it's" in the README should be "its": https://brians.wsu.edu/2016/05/19/its-its/

Edit: Actually, the short token is more than just a token ID - it's also required, right? Perhaps replace:

   When we receive an incoming request, we search our database for hash(long_token)

with:

   When we receive an incoming request, we search our database for hash(long_token) and short_token. A token can be blocklisted by its short_token.

So I think I prefer "short token" to "token id" but perhaps there is a better name for it. If it did get renamed it would probably make sense to rename "long token" as well. I'll defer to experts on this.

seveibar11• 4 years ago

fixed thank you :) - and great suggestion!

edit based on your edit: Added your recommendation to the README, thank you!

benatkin7• 4 years ago

My suggestion was blocklist which looks a lot like blacklist because only one letter is different. Perhaps "deny list" would be better. It's what Apple is using. https://developer.apple.com/news/?id=1o9zxsxl

benatkin7• 4 years ago

One more suggestion - replace

A token can be blocklisted by its short token.

With:

An API Key can be blocklisted by its short token.

anm89• 4 years ago

Why base 58? I've never even heard of base 58.

jffry2• 4 years ago

The linked README links a draft RFC [1], the introduction section of which contains an explanation.

[1] https://datatracker.ietf.org/doc/html/draft-msporny-base58#s...

NavinF• 4 years ago

Wow how did I never hear about this? I only read the first 2 paragraphs under Introduction, but it addresses all the qualms I have with base64 and its many variants.

__sy__2• 4 years ago

from the RFC:

Base58 is designed with a number of usability characteristics in mind that Base64 does not consider. First, similar looking letters are omitted such as 0 (zero), O (capital o), I (capital i) and l (lower case L). Doing so eliminates the possibility of a human being mistaking similar characters for the wrong character. Second, the non-alphanumeric characters + (plus), = (equals), and / (slash) are omitted to make it possible to use Base58 values in all modern file systems and URL schemes without the need for further system-specific encoding schemes. Third, by using only alphanumeric characters, easy double-click or double tap selection is possible in modern computer interfaces. Fourth, social messaging systems do not line break on alphanumeric strings making it easier to e-mail or message Base58 values when debugging systems.

godmode2019• 4 years ago

Very cool,

I use something similar but a fixed length for the prefix and uppercase

Basically a copy of stripe tokens

phineyes3• 4 years ago

I created a similar project recently! Main difference is that I use base64 to encode unique Snowflake IDs, which are timestamped: https://github.com/hopinc/pika :)

benatkin7• 4 years ago

In your example, the token contains the timestamp.

With prefixed-api-key, the hash of each token is stored in the database, so the timestamp can easily be added there.

Most of the time I don't see the utility in the client having the timestamp, outside of a scenario where you have a third party validate on their own (e. g. JWT w/ RSA keys). The best way you see if a token has expired is by trying to use it.

phineyes3• 4 years ago

Yeah, pika is more of an ID system vs. prefixed-api-key which seems to be more oriented around just API keys, which makes sense. However, advantages of timestamps in IDs are that you can create unique IDs based on the current timestamp; bits which, in other ID systems, are usually just random - which feels like a waste imo. Also, knowing when a resource was created when debugging is very helpful.

lefrenchy• 4 years ago

I wasn’t too sure based on the docs:

What parts of tokens does the server store in the Database (everything but the raw long token I think?)

What is sent to the consumer, and what do they need to send with the request?

seveibar11• 4 years ago

Yep you've got it! The consumer gets the full api key containing the prefix, short token and long token, e.g. "mycompany_BRTRKFsL_51FwqftsmMDHHbJAMEXXHCgG". The database stores the short token, and a hash of the long token.

tetch• 4 years ago

Nice work! May I inquire as to the design decision behind storing a hash of the long token server-side for verification of the long token, versus digitally signing the short token plus a nonce and including this signature in the API key (instead of a long token), as some other API key schemes do?

Both are valid approaches of course, I'm just interested to hear your thoughts on the relative tradeoffs.

seveibar11• 4 years ago

The signature approach is interesting because it couples the short token the api key, so they're a pair rather than totally independent. Introducing a signature changes the math to create the shortest possible API key with at least 128 bits of entropy (that are not stored server-side). But it's worth looking into!

kevincox• 4 years ago

we search our database for short_token and hash(long_token).

Sounds like some salt should be used here. Search the DB for short_token, then use the discovered salt to check the "password".

wccrawford• 4 years ago

  // Store the key.longTokenHash and key.shortToken in your database and give
  // api.token to your customer.

I'm pretty sure that should be "key.token" instead.

seveibar11• 4 years ago

thank you!

miller_joe• 4 years ago

Hey, cool idea. I like it. I made a Go implementation for fun/practice - https://github.com/joemiller/prefixed-api-key

Show HN: Prefixed, dual-token, base58 encoded API Keys

Related Stories