Creative usernames and Spotify account hijacking

http://labs.spotify.com/2013/06/18/creative-usernames

by e1ven • 13 years ago

281 79 13 years ago

labs.spotify.com

This seems odd. I mean, if their code was properly modular, they would have just one place where they "fetchUserIdByName(userName)", which returns one user ID or null if it's not used yet.

When a new user is created, it then gets assigned a unique user ID. The email address is assigned to that user ID.

Then, if they do a password reset on user = "bigbird", it should do the exact same lookup to find the email address.

The security bug was not really about having an improper function to do unicode translation. It was more about having different functions for the same check, simply because they were in different parts of the code.

Modular code is just so much better on all fronts, including security.

eli2• 13 years ago

Easy to implement in a small project. Not so easy in a large enterprise project likely comprised of multiple interconnected systems.

TillE• 13 years ago

Not a good excuse. There are plenty of enormous projects that provide only one basic interface to a bit of information. See every operating system API for examples.

dan_manges2• 13 years ago

Based on their description of the bug, it sounded like the code was modular, but they called the function twice: once when the password reset request was generated, and again when the link in the email was clicked.

  However, when the link was used, canonical_username was once again applied

So after they sent the password reset link, they called "fetchUserIdByName" again, but they passed in a username that had already been canonicalized once. Because of this bug, I wonder if password resets worked at all for users with unicode characters in their names.

mmahemoff2• 13 years ago

If you're saying canonicalise(canonicalise(name)) is not the same as canonicalise(name), that's going to be seriously bug-prone. Idempotence ftw.

shawnz• 13 years ago

This is what I thought at first. If it's the exact same check (which it should be), why is there any possibility of the answer being different? But the real "bug" here is that they were treating the canonicalized username as if it were just as good as the original username, which is only true if the adjustment function is idempotent as they say.

Another possible solution would be to assume the username given to the password reset form is already canonicalized (which would be necessarily true, as far as I understand).

EDIT: However this wouldn't solve the other bug that's been discovered here, which is that "ᴮᴵᴳᴮᴵᴿᴰ" is canonicalized differently than "BIGBIRD", thus defeating the purpose of canonicalization (for that particular case) in the first place.

brown9-22• 13 years ago

Sounds like they are using the username as the key in their DBs, which sounds like the ultimate case of any pain:

Could the method for computing canonical usernames based on nodeprep.prepare() be salvaged? If not we would be in trouble since we use canonical usernames in various databases so that changing how to derive them in a non-backwards compatible way would be quite costly.

tantalor4• 13 years ago

Not necessarily. It their canonicalization function were idempotent (e.g., the identity), then this database scheme would work well.

How else do you map username to user id?

tantalor4• 13 years ago

What you're talking about has nothing to do with modularity. You're thinking of DRY. They having nothing to do with each other.

DRY tells us the code for looking up a user id by user name should be written in only one place, not that the code should be called from only one place.

The mistake was assuming the name->name function was idempotent, because it wasn't.

You are right to suggest using a name->id function instead. It would not suffer the same problem because the canonical name should not be stored... it's an implementation detail!

Confusion• 13 years ago

  They having nothing to do with each other.
  DRY tells us the code for looking up a user id by user 
  name should be written in only one place, not that the 
  code should be called from only one place.

If you write 'canonicalize(username)' in eight different places, you are not being DRY. If you need to write 'canonicalize(username)' in eight different places, your code probably doesn't separate responsibilities properly across separate modules.

As such, they have a lot to do with each other. After all, if you call code from more than one place, you are writing the calling code in more than one place. Lack of DRYness is about the fact that you are doing so. Lack af modularity is about why you need to do so.

Jhsto• 13 years ago

I'd like to point out that even though not every website accepts unicode characters on registration, some software such as vBulletin let the administrator/user to replace the username with such characters once the user is signed up.

You may not create as much havoc as in the original post, but some level of confusion at least.

pilsetnieks2• 13 years ago

I'm confused about why they even had the need for canonicalising the usernames for whatever purpose? Why couldn't they just have stored and used them just as they are?

dan_manges2• 13 years ago

They gave one reason in the post. They wanted usernames to be case insensitive, so that if there's a user named BigBird, somebody else can't sign up as bigbird. Case insensitive usernames are also helpful to minimize support issues when somebody forgets the exact case they used when they created their account.

drivebyacct24• 13 years ago

Am I insensitive in wondering "Why not just limit it to lowercase, ASCII chars"?

yathern• 13 years ago

Because, then there can be multiple accounts that are variations of BigBird, and all annoying to differentiate for users. Such as Bigbird bigBird BigBIrd. Combind with this, the fact that you have unicode characters, and it makes it very easy to 'spoof' as another user by simply changing the letter 'o', to a unicode circle, or various other tricks.

jessaustin• 13 years ago

If they did as you suggest, the specific account-stealing flaw they had wouldn't happen, but since many unicode points have very similar glyphs, there would still be "copycat" accounts. That is, the strings "Oscar" and "Οscar" appear very similar (if one has the proper fonts installed), and one user could therefore pose as another.

It's true that many sites don't care about this, but I don't fault Spotify for trying to prevent it.

pilsetnieks2• 13 years ago

It looks to me as if the prevention of this caused more problems than it solved.

United857• 13 years ago

A similar issue exists with international domain names.

https://en.wikipedia.org/wiki/IDN_homograph_attack

Groxx2• 13 years ago

it's a different problem, though to some degree they stem from the same underlying issue (similar but different characters).

Esifer• 13 years ago

Why does Unicode threat Omega and Ohm like different characters?

eli2• 13 years ago

Why does HTML have a <strong> and a <b> tag?

sleepyhead• 13 years ago

<strong> is emphasis/semantic and <b> is visual/style.

drrotmos• 13 years ago

Because one is a greek letter, and the other is a unit of resistance. The fact that they're represented by the same symbol is irrelevant.

threedaymonk• 13 years ago

If that were the case, we'd need separate codepoints for every letter used as a unit of measurement, from A for ampere onwards. In fact, it's just there for legacy reasons, as pointed out elsewhere: the convention for units of measurement is to use normal letters, regardless of whether those are Latin or Greek.

duskwuff• 13 years ago

Because they have rather different meanings. Capital omega (Ω, U+03A9) is a Greek letter, with the lower-case form ω; ohm (Ω, U+2126) is a symbol used in electrical engineering with a related symbol "mho" (℧, U+2127).

tmarthal• 13 years ago

FYI, that all units measuring physical properties named after scientists use capitalized letters for their abbreviation[1]. So the ohm (named after Georg Ohm) is abbreviated as the uppercase omega, no idea why they are different unicode values, since they do not have different meanings.

Note that omega was probably used so that the 'O' wouldn't be confused with '0', e.g. 4O would be confusing, but 4Ω is not.

[1]The tesla is abbreviated 'T', joule is 'J', etc. etc.

mmahemoff2• 13 years ago

Unicode has lots of characters that look alike, because, well, lots of human languages ended up with similar-looking characters.

Even English speakers will easily confuse 1,I, and l, depending on how they're represented by the browser. And 0/O.

For more fun, try drawing any shape on http://shapecatcher.com/ and see all the similar-looking Unicode characters.

sgarman• 13 years ago

This is a really fun way to explore unicode, thanks.

moskie• 13 years ago

Wikipedia's article on Ohm actually covers this!

http://en.wikipedia.org/wiki/Ohm#Ohm_symbol

"Unicode encodes the symbol as U+2126 Ω ohm sign, distinct from Greek omega among letterlike symbols, but it is only included for backwards compatibility and the Greek uppercase omega character U+03A9 Ω greek capital letter omega (HTML: Ω Ω) is preferred."

And from the Unicode Standards doc that is the source for that section:

"Greek Letters as Symbols: The use of Greek letters for mathematical variables and operators is well established. Characters from the Greek block may be used for these symbols.

For compatibility purposes, a few Greek letters are separately encoded as symbols in other character blocks. Examples include U+00B5 µ n the Latin-1 Supplement character block and U+2126 Ω in the Letterlike Symbols character block. The ohm sign is canonically equivalent to the capital omega, and normalization would remove any distinction. Its use is therefore discouraged in favor of capital omega. The same equivalence does not exist between micro sign and mu, and use of either character as micro sign is common; for Greek text, only the mu should be used."

afandian• 13 years ago

Is there a separate symbol for A?

gkoberger• 13 years ago

Reminds me of the early GMail bug, where you could sign up with "john.smith", "jo.hn.smi.th" or "johnsmith", but all mail was delivered to the period-less version:

http://arstechnica.com/uncategorized/2006/01/6022-2/

knome• 13 years ago

That isn't a bug. Even the article you link notes that it is a feature, though they seem for some reason incredulous.

I've always liked that they do this on my accounts.

Maxious• 13 years ago

The link agrees - it's a feature not a bug and one that considered security:

As it turns out, even though Gmail will act as if there is no period in a username when delivering mail, it will not permit users to register accounts whose only difference to other account is a period. That is, if bob.jones is registered, bobjones cannot get an account (he could get bobjones2, obviously).

praseodym• 13 years ago

You could (and still can) only register one of those addresses. Mail from the other two will be delivered to that address.

So, it's not a bug but rather a pretty nice feature: ever seen a handwritten email address with a period squeezed in there?

a-nikolaev4• 13 years ago

I don't see any real reason to rely on idempotence.

They could simply store two names: One is provided by the user (verbatim), and the second is its reduction to lowercase letters and digits (canonical). For all internal logic, they could use only the canonical name, and use the verbatim name in the front-end to make the user happy.

Lower casing has the key property of being idempotent, i.e., that applying it more than once has no effect: x.lower() == x.lower().lower(). So if a username gets passed from service to service and you want to make sure it is in canonical form you can safely apply .lower() and if it was already in canonical form there is no harm done, and it is easy to stay safe.

Apparently, they thought that it's ok to use verbatim and canonical names interchangeably, relying on idenpotence property of the XMPP function.

It looks like a huge design error.

readme2• 13 years ago

They could simply store two names

Presumably you mean in the database? I don't see a reason to keep a copy of the lower() transformation of a string when it is incredibly cheap to transform a small string to lowercase.

What exactly is the point of that? I would just call lower() as needed, personally.

a-nikolaev4• 13 years ago

Oh, my point was that two kinds of names should not be used interchangeably. To store or not the .lower() is a matter of taste.. (Personally, I would store both names, just to avoid wasting computing time)

gwright• 13 years ago

If you are testing for uniqueness of the canonical name you don't want to have to recompute the canonical name for all your existing records.

willvarfar2• 13 years ago

Canocialisation functions are, by definition, idempotent.

Their canocialisation function, which is a standard one, was broken by subtle changes in Python 2.5, but worked previously.

a-nikolaev4• 13 years ago

Yes, I agree that it sounds great. But, it does not mean that you really have to use this property. It was not necessary for the task, this is the point..

Well, in general, modular code should not make assumptions about other parts of the program (when possible).

You know, if the function is idempotent by definiiton, it does not mean that its implementation is. Unicode is changing too, new symbols are added.

drivebyacct24• 13 years ago

So is there a need for a TRULY idempotent equivalent of XMPP's nodeprep? Or one that handles more Unicode points? Or is it a calculated decision to support Unicode 3.2 points only? (Sorry for the nooby questions, but this was very interesting and I don't know a lot about Unicode)

acjohnson55• 13 years ago

Correct. And it would seem to me that such a function should probably be in its own separate library, or perhaps the standard library. It seems like a fairly basic need.

lancestout• 13 years ago

The decision to only support Unicode 3.2 is simply because the StringPrep framework [1] (which XMPP's nodeprep and various other protocols use) is forever tied to that version of Unicode.

Current work is on the PRECIS framework [2] which uses the metadata for Unicode code points to determine how to handle them during canonicalization instead of relying on a hard coded set of mapping tables. There's still a lot of work to be done, mainly to review that the process works reliably and doesn't introduce subtle new issues. Peter Saint-Andre (one of the authors of PRECIS) has just started on a Python tool for testing how a given version of Unicode is handled by PRECIS (https://github.com/stpeter/PrecisMaker).

[1] https://www.ietf.org/rfc/rfc3454.txt

[2] https://tools.ietf.org/html/draft-ietf-precis-framework-08

drivebyacct24• 13 years ago

Great information, even cooler to see someone's got some code working alongside it, I might have to adapt it to Go if it's fairly reasonable to understand given my relative lack of experience with Unicode (heh as I'd mentioned and is probably obvious given your knowledge on hand).

waster• 13 years ago

A fascinating account.

tantalor4• 13 years ago

So if a username gets passed from service to service and you want to make sure it is in canonical form you can safely apply .lower() and if it was already in canonical form there is no harm done, and it is easy to stay safe.

Why?

Suppose you only pass the original name around instead. Then you don't require your canonicalization function to be idempotent, which might be good in your case since it wasn't.

antocv• 13 years ago

Because they wanted "BigBird" to be the same as "bigbird" or "BIGBIRD". Hence if you wanted to rely on a userId, you would still need to ensure all the variations of BiGbirD map to the same userId.

NKCSS• 13 years ago

I think they could have handled the 'reward' a bit better :P

"In this case the two users who posted to the forum where actually rewarded with some Spotify premium months."

I'd say: Premium lifetime memberships would be better :)

robmcm• 13 years ago

Yeah, seeing as before they had the option to log into anyones account, affectivly the same as a lifetime membership.

Sami_Lehtinen• 13 years ago

Nothing new at all, even my stockbroker had similar kind of bug. There should be only one function which handles usernames. When same thing is implemented differently all over the source, this is exactly what happens.

I have seen also much worse solutions. Where actually giving username, logs you in (sets logged in session cookie) and then prompts for password. When you enter invalid password you're logged out. If you give username, and then change url, you're in. Business as usual. When you test it, it works. Username + right password = ok, Username + wrong password != ok. Tests passed, and that's it.

paul_f• 13 years ago

We require an email address as a user name. Does this get around the problem?

troels• 13 years ago

Depends on how you validate an email address.

jrochkind13• 13 years ago

Very interesting post -- dealing with unicode gets tricky. Because dealing with a full character repertoire is tricky, not because unicode does it poorly (unicode actually does it well).

I wonder if Unicode UTR#30 would be an alternative (more reliable?) method of idempotent normalization? http://www.unicode.org/reports/tr30/tr30-4.html

Or maybe the twisted algorithm really is UTR#30, just not labelled that?

As far as I can tell, UTR#30 did not make it to formally being part of the unicode spec, for reasons I'm not entirely clear on -- it is nonetheless quite useful, and this case is an example. Solr for instance still uses it. (http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#...).

It might be a pain to find code implementing UTR#30 in your language of choice though (I am not sure if it's part of current ICU libraries or not).

It's also worth pointing out, that in addition to this kind of 'folding' of different-but-look-the-same graphemes, in this sort of use case you ABSOLUTELY need to do byte normalization as per UAX#15 http://unicode.org/reports/tr15/ . Probably NFKC for this sort of use case.

jrochkind13• 13 years ago

Huh, and another alternative would be using one of the unicode collation algorithms for normalization -- which, unlike UAX#15, did make it beyond draft stage to be an official part of the unicode spec.

http://www.unicode.org/reports/tr30/tr30-4.html#_Toc23

nodata• 13 years ago

it is probably good to... treat “BigBird” and “bigbird” as the same username.

Erm...

Groxx2• 13 years ago

How about "building" and "buiIding"? or "nodata" and "ｎodata"? You probably don't want people running around claiming to be admins because their name looks the same.

Similarly, do you really want to do tech support when someone forgets that their username is "nodata" instead of "Nodata", since their phone auto-capitalized things when they signed up? It happens. And how do you know they're "Nodata" and not "nOdata" or "nOdAtA"?

dhtbccnhmdhm• 13 years ago

Why not to make canonization while the names differ?

canName=canonical_username(name); for(canName1=canName,i=0;canName1!=canName;i++){ canName1=canonical_username(canName); if(i>=treshold){ stop_registration(); break; } }

Creative usernames and Spotify account hijacking

Related Stories