King – Man + Woman = King?

https://medium.com/p/king-man-woman-king-9a7fd2935a85

by flo_hu • 7 years ago

256 103 7 years ago

JD5572• 7 years ago

At first I thought "What's the big deal? You also remove the query from recommender systems, for example. It's an obvious uninteresting result!".

But then I read the linked article [Nissim 2019] and it all became much clearer with te example in the title: "Man is to Doctor as Woman is to Doctor". If you remove "Doctor" from the results, you'll get "Nurse" instead, not because the dataset/society/... is biased, but because you inserted the bias in your model!

With this small "optimization" (and others presented in the article, such as the "threshold-method" and hand-picked results from the Top-N words), it's trivial to use the anology method to show that any dataset is biased, since you are filtering the unbiased results!

One a side note, I wish articles like this were more popular. I find that it's really easy to use AI techniques "the wrong way" and there's are a lack of articles pointing to common pitfalls (and, to make things worse, there are a lot of blog posts doing things wrong, which only validates bad methodology).

[Nissim 2019]: https://arxiv.org/pdf/1905.09866.pdf

sooheon2• 7 years ago

Wow, the Manzini et al. paper is particularly egregious, because they accidentally swapped "caucasian" and "black" in their actual query and still got the sensational word associations they wanted.

dmurray• 7 years ago

Yeah, that's more or less academic fraud.

tgb3• 7 years ago

The other reason to not exclude King is that it's pretty likely that Queen is the closest word to King. Then King - X + Y = Queen is just saying that X and Y are close to each other, not an interesting result.

novaleaf• 7 years ago

your 2 sentence explanation is better than the entire article!

Tarq0n• 7 years ago

Considering the very high dimensionality of the space, it's not as obvious as you think. Consider the noise inherent in the word 2vec negative sampling method as well. Another word could very well end up closer to "king".

derefr• 7 years ago

From my perspective (linguistic anthropology), they’re not actually all that close. Most historical “queens” (which have that label applied to them by modern English speakers) were not rulers (and we have a separate term, “queen regnant”, for that) but rather the gender dual to the male “royal consort.” It was only in recent history where you see examples of “equal-opportunity” monarchies that could have either a male or female monarch of equal power, and thus usages of “queen” to denote those monarchs.

Thus—given that we’re defining words based on their centroids of usage in a historical corpus—if a woman is a monarch of a kingdom, “king” is a tighter historical fit to describe her role than “queen” is.

dllthomas3• 7 years ago

I wonder if there's anything interesting in normalizing Y - X somehow.

petters2• 7 years ago

It is not clear that they are close.

I did the same thing with Japanese and Western movies, and the result was not due to the movies just being close. See https://strandmark.wordpress.com/2017/07/08/the-vector-from-...

stult• 7 years ago

I think there are two ways to look at excluding input vectors from the output. You can look at it as cheating/bias, or you could say the method explicitly excludes a specific subcategory of otherwise valid results (i.e., perfectly equivalent analogs, where X:Z == Y:Z... not sure if there is a term of art for that). Which to me seems like a perfectly legitimate limitation on the algorithm, as long as it is explicitly acknowledged and handled.

But I think you are absolutely correct that very few articles explicitly acknowledge and handle these types of common pitfalls. Especially more intro-level articles. Personally, I'd love to read more about effective strategies for ensembling different algorithms to work around some of those pitfalls. For example, the OP points out that Word2Vec underperforms on lexical semantics. I wonder if you could use a different algorithm that performs well on lexical semantics but poorly on other categories as a supplement, with some third algorithm deciding which to use in any given case.

flo_hu8• 7 years ago

I would also agree with you that it is fine to add additional rules to improve the outcome, but than it shouldn't be made clear in the way the result is presented (as you say, that rarely happens in intro-level tutorials/courses).

Your last point sounds like a cool idea! Using those more in-depth metrics to find weaknesses and see if other, complementary algorithms can fill the gap.

bjourne3• 7 years ago

The dataset is biased. Otherwise woman~nurse wouldn't have been the second hit for the query. Had the dataset not been biased it would have been able to produce the analogy "woman is to doctor as man is to nurse". See table 4b where they perform that experiment.

I think it is also important to note that this is an argument over which of two methodologies is best. The answer likely is that it depends on the use case. Researchers are not being accused of using underhanded methods, manipulating data or cheating.

Diggsey• 7 years ago

Bias is not a boolean quantity. The underlying data set is somewhat biased, and the "trick" being used greatly exaggerates the impact of that bias.

Majromax2• 7 years ago

The dataset is biased. Otherwise woman~nurse wouldn't have been the second hit for the query. Had the dataset not been biased it would have been able to produce the analogy "woman is to doctor as man is to nurse".

I think that this hypothetical example would be evidence of bias. "woman:doctor :: man:nurse" as an analogy only works because of gendered implications of 'doctor' versus 'nurse', so if that were a robust output of the model it would imply that the word2vec found that gender was of great significance in the manifold near 'doctor'. (The analogy would just be applying a negative weight to the gender basis, but that's mathematically fine even if not customary.)

6gvONxR4sf7o• 7 years ago

For others reading, there are still important biases these models pick up (for a variety of reasons I won't go into). The takeaway here is that some of the evidence you've heard about isn't really evidence, not that the biases aren't there.

thrwaway3873• 7 years ago

but in analogies, a:b::c::___ (a is to b as c is to what), having d be the same as b is not allowed. Such as, I believe, on a Miller Analogies Test.

In a mathematical context, if you have something like this: 2:0::4: (two is to zero as four is to what?) then acceptable answers might be 2 (rule is: subtract 2), or maybe it could be 1 (take half, subtract one). But it couldn't be zero. (rule can't be "multiply by 0").

I could be wrong but I believe that's how it works.

dragonwriter2• 7 years ago

but in analogies, a:b::c::___ (a is to b as c is to what), having d be the same as b is not allowed

Particular tests may adopt such a rule, but this doesn't apply to analogies in general. Particularly, consider the case where Alice and Carol and both daughters of Bob: Alice:Bob::Carol:Bob is a perfectly valid analogy; a:b::c:b is commonly phrased along the lines of “a and c are similarly situated with respect to b”.

Jenz• 7 years ago

I find that it's really easy to use AI techniques "the wrong way"

NLP-wise, is there "a right way"?

JD5572• 7 years ago

Just because there's not a "right way" doesn't meant that there aren't a lot "of wrong ways" ;)

Actually, some years ago I bumped into a similar problem to the one discussed, where someone wanted to use NLP to show gender bias in a dataset, and hit one of the common pitfalls that I mention (I hope that I don't start a flamewar by sharing this story):

Here's what they tried to do: 1. Fetch a list of ~800 atendee names from a Portuguese tech conference (it had an official API with user profiles) 2. Download a dataset most common male/female names for newborn babies in Portugal and America for the latest 3 years 3. Train a naive bayes model on the downloaded dataset and use it to classify the antedees into male/female

After doing that, the algorithm returned something like "8 female attendees and 792 male antendees".

I found this particularly strange (considering that I knew more than 8 women that attended on previous years), so I took a peek at the antendee dataset and found that: - There were some users using a fake name (including one organization account) - There were certainly more than 8 female antendees, and at least 6 were named "Inês" (female name)

After discussing this with the ones involved, we found the problem! - The dataset was not being normalized (it was being trained with "Ines" and tested with "Inês") - The naive bayes[1] implementation used, when faced with a completly new input, outputed the most common class of the training dataset

In the end, the final result was closer to "80 female, 705 male, 15 unknown", which is a much more believable result (closer to the typical distribution of Software Engineer students in Portugal).

Note that the author wasn't trying to deceive anyone, he just tripped on some common pitfalls (forgot to normalize the data and used an off-the-shelf implementation without looking into the details).

[1] There was only one attribute, so implementing this without using a naive bayes library was actually easier and produced the correct results.

raverbashing3• 7 years ago

The author seems to have just learned that most machine learning algos are not "free lunch" but there's a good amount of poking and tuning and some "cheating" needed.

It's part of the learning curve, and I spent quite some time to learn the basics

flo_hu8• 7 years ago

I'm fine with the free lunch thing. But here the cheating is done on the level of how people present the capabilities of the tool. If you ask the algorithm how "SHE is to LOVELY as HE is to X", the reported answer (Bolukbasi 2016) was "BRILLIANT", which in this case suggests a heavy gender-bias. But what the algorithm actually gives for X is: "LOVELY". The authors justed picked the 10th example in the list without clearly stating it.

raverbashing3• 7 years ago

"Cheating" (in practice) usually means "embedding problem/domain specific quirks"

In the "King" example, you're adding and subtracting two words that are probably very close already, so if you want to find "something else" besides itself, you need to exclude it. For some problems it might make sense, for some others it might not.

yorwba2• 7 years ago

The authors justed picked the 10th example in the list without clearly stating it.

That's not an accurate description what Bolukbasi et al (2016) [0] did. In particular, they do not list x close to lovely + he - she and then pick arbitrarily from that list. Instead, they explicitly reject that approach (see appendix A), because they're looking for pairs of words that are maximally gendered. They do that by finding x and y such that the angle between x - y and she - he is minimized. Since the task they're solving is different, you can't fault them for getting different results.

[0] https://arxiv.org/abs/1607.06520

willvarfar• 7 years ago

Just yesterday NHers were warned to stop using 'free lunch' in conjunction with AI: https://news.ycombinator.com/item?id=20436205

pfortuny• 7 years ago

So, in some sense we are back to "eliminating outliers"... using an ad-hoc definition of "outlier".

Does not look very scientific to me.

flo_hu8• 7 years ago

Exactly! I think that was part of the problem for many of the examples that turn out to not-really-work. People pretend they let the work do by an algorithm, but then hand-pick from a list of somewhat close candidates. Which of course happens with a hypothesis (and thereby desired outcome) in mind.

Ensorceled• 7 years ago

That's fine if you're using it to make business decisions or embedded in a chatbot, but if you're using your new NLP algorithm to "discover" latent social biases and then make commentary .... then a "good amount of poking and tuning" is, at best, academic fraud.

manuka• 7 years ago

Who thinks here that:

"King – (Man + Woman)"

and

"(King – Man) + Woman"

Must have different outcome? I do. No wonder Word2Vec sucks! Words are not elements of vector space :)

bennofs• 7 years ago

King - (Man + Woman) is the same as King - Man - Woman which is not the same as your second example at all.

toxik• 7 years ago

The idea is that king k and queen q are both near some location r with a small displacement so k = r + dk and likewise q = r + dq. Then also so around s for man m and woman w, so m = s + dm and w = s + dw. The idea then is that dk = dm and dq = dw, this is the ”semantic structure.” The displacement is the same. So k - m + w = r + dk - dm + dw = q. It would have been cool, but it seems what really happens is queen gets embedded close to king, and the nearest word is just queen. I guess it does show that dk = dm and dq = dw though, but it could also be that dk = dq, and dm = dw.

pure-awesome• 7 years ago

Subtraction is not associative. One cannot just move parentheses around like that with numbers either:

5 - (2 + 3) =/= (5 - 2) + 3

The same holds for elements of a vector space.

raverbashing3• 7 years ago

You are correct

I was about to say you were wrong, but you are correct, and it is a bit unintuitive why - https://www.quora.com/Is-vector-subtraction-associative

Retric• 7 years ago

The first term may be more accurate than you might think. Elizabeth I of England ruled as Elizabeth Rex where Rex is Latin for "King" and Regina is the female term.

Technically in England Queen is an ambiguous term, queen regnant is the actual ruler, queen dowager is the widow of a king etc. But, again several European an other female rulers used the male term.

Hatshepsut crowned herself Pharaoh and maintained an elaborate legal fiction of maleness, because ya know she was in charge and could do what she wanted.

Hungary had two female kings Mary of the House of Anjou, and Maria Theresa.

Etc etc.

flo_hu8• 7 years ago

Good point! I would see this rather as yet another argument for why you should simply give the actual output of the NLP algorithm.

So if people actually do the calculation King-Man+Woman and it comes closest to King, than they should report "King-Man+Woman~=King" and not "King-Man+Woman=Queen" (only because that's what they expected).

feanaro2• 7 years ago

In the end, it doesn't matter what transformation you do though, as long as you do it consistently and/or not in an ad hoc manner. If excluding the original term always leads to useful results, it is a useful transformation.

The problems materialize when you're just cherry picking for results.

Sean1708• 7 years ago

To be honest, I think the idea that we should expect ML algorithms to give a single, certain answer is misguided. I would expect the output from this algorithm to be "King - Man + Woman = King (90%), Queen (83%), Prince (70%)" or something like that, i.e. a list of answers with some measure of how "good" those answers are. Then again, I work in a field that doesn't really have categorical answers so maybe I'm missing something obvious.

seba_dos1• 7 years ago

Poland had King Jadwiga, although in common language people still call her a queen (as she actually became one a few years after she was crowned).

YeGoblynQueenne• 7 years ago

See also: Drag kings (https://en.wikipedia.org/wiki/Drag_king). (Usually) female people called "king".

johnchristopher• 7 years ago

https://miro.medium.com/max/700/1*tFmq8FUI8SzxKKqo_kgmpQ.jpe... I think the AI should spend more time in a gender equality classroom (and probably its maker too). :)

roel_v• 7 years ago

...

Did you even read the article? Its whole point is that sensationalist conclusions like yours are wrong.

tgb3• 7 years ago

I've pondered making a video game based off Word2Vec. User collects words from conversations with NPCs and had to combine them to get new words which they use in conversation or maybe as physical items to cross barriers, etc. Not quite sure how it would work except that it would probably devolve into just random guessing. Still, I'd love to see it tried.

duckmysick• 7 years ago

While not based on Word2Vec, there's a puzzle game that uses words to craft and manipulate rules of a physical world: Baba Is You. https://hempuli.com/baba/

martindbp• 7 years ago

I started playing around with word embeddings in SpaCy for finding synonyms, antonyms or any kind of word similarity that could be used to cluster a list of words into different conceptual groups. So far, the results doesn't seem to correlate very well with what I would consider "similar". Hopefully combining it with tools like WordNet and lemmatization + Levenshtein distance can get me closer to something useful though.

nerdponx• 7 years ago

Word vectors like this only work if the two words you care about appear in similar contexts in the corpus it was trained on.

So the assumption is that words from similar context should be similar. But you're always going to miss out on some words that are similar but do not appear in similar contexts.

feanaro2• 7 years ago

If the words are really similar in meaning, you should still be able to arrive at a useful result using some graph manipulations.

The words might not appear in a directly shared context, but given their semantic similarity, they should share more contexts of distance 1 (or something along those lines) than an arbitrary pair of words.

derEitel2• 7 years ago

What happens if you get the latent space of king, do no algebra and return the outcome with king being excluded? In case it's queen, then the author is correct and these examples are highly misleading. In case it's something like prince, Lord or ruler of the seven kingdoms the latent space algebra would be suitable imho.

Also, if we think about it in terms of decision manifolds, it seems the distance between queen and king is too large for the simple - man + woman to have an effect. Why not scale that substraction, so it leads to a change in predicted class without removing king? But of course finding a justifiable weight would be hard..

Erlich_Bachman• 7 years ago

Yes, the result seems entirely predictable and logical to me. Even using normal human logic, even if the amount of meaning that the whole term "king" contains is 100%, the "male" part of that meaning would answer for maybe what, ~20%? The rest would imaginatively be filled with things like "top executive in an organization", "power by family relations, not by democracy", "fancy clothes", "chess piece", "ceremonial post more than actual decision making in some countries", "feudalism", etc.

philh• 7 years ago

In one model I found online[1], the closest word to "king" was "kings" (0.71) and the second closest was "queen" (0.65).

[1] http://bionlp-www.utu.fi/wv_demo/ (making sure to select the English model)

derEitel2• 7 years ago

If we eliminate king from the result list I would assume it's plural is removed as well. Bummer, that would mean the latent space algebra in this example has no effect whatsoever..

slx26• 7 years ago

I don't know. The problem here might be human expectations. As the article says, word2vec sometimes is "sold" with these kinds of analogies that seem so intuitive. So we come to expect the system to behave intuitively. And in many cases it does.

But, what are you really doing there? Well, you are operating with vectors. And what do those vectors represent? Not the semantics about the words themselves. The vectors encode the context in which they appear. They encode what other words tend to surround them.

And there are more limitations. The most obvious one is polysemy. But even without polysemy, different words might have more or less "diffuse" representations. The more technical, precise and uncommon a term is, the less "diffuse" its vector is. King and queen, man and woman are far from precise, technical or uncommon terms. And therefore, there's a lot of "diffusion". Again, the problem here is our expectations versus the results we see in practice. But the models are what they are. And they are not magic.

SiempreViernes2• 7 years ago

The titular example is very often sold as something very close to the semantics; so the problem is indeed in the expectations, expectations created by how word2vec embeddings are often presented as vectors that magically understand the semantics.

mr_crankypants4• 7 years ago

A related thing that amuses me: One of the (less popular†) benchmarks for embedding models is their performance on a TEFL word analogy test. At least as of the last time I perused the literature, which was a few years ago, there was one algorithm with which I've seen anyone report really high scores on this test. For all the crowing about how great Word2Vec is at finding analogies, it wasn't SGNS or CBOW. Not GloVe, either. It was boring, unsexy, 30-year-old, doesn't-even-have-a-strong-theoretical-foundation latent semantic analysis.

† Because it doesn't measure performance at something people typically want to do with these models in real life - there's a strong "party trick" component to the word analogy stuff.

Der_Einzige4• 7 years ago

Topic models are going to get blown the hell out by UMAP and other manifold based dimensionality reduction algorithms

mr_crankypants4• 7 years ago

I haven't been following the literature too closely lately - have there been promising results in using it for topic modeling or semantic vector modeling?

My own sense is, the real problem that the space needs to contend with isn't that the math isn't elaborate enough. It's that we're still waiting for someone to come up with a really good way to deal with polysemy, and with multi-word phrases that form a single semantic unit.

scotty792• 7 years ago

King - tomato + potato = Queen

flo_hu8• 7 years ago

No. That's of course still "King" :)

(but sure one could also pick queen, prince, royal form the list...)

Just tested it here: http://vectors.nlpl.eu/explore/embeddings/en/calculator/#

And it gave me 0.63 King, 0.6 Prince etc...

scotty792• 7 years ago

So =Prince because you should exclude King similarly how you exclude to get Queen in original example.

red75prime• 7 years ago

Top 5: monarch, kings, queen, crown_prince, prince. From http://bionlp-www.utu.fi/wv_demo/

DoctorOetker• 7 years ago

A bit off topic: does anyone know where I can easily download pretrained simultaneously trained separate word and context vectors? that is before they are added together to get a single set of vectors?

zerubeus• 7 years ago

Stop publishing to medium for God’s sake!!!

lsorber• 7 years ago

What should we be publishing to?

wl• 7 years ago

Essentially anything else.

anchpop• 7 years ago

Medium wants me to start a $5/month to read this article. If I pay does the author see any of that money? Or is it just medium capitalizing off their writing

flo_hu8• 7 years ago

I removed my blog post from medium's distribution. So it should now be freely accessible! https://blog.esciencecenter.nl/king-man-woman-king-9a7fd2935...

program_whiz• 7 years ago

A subtle point not really mentioned or brought out. These vectors are 300 dimensional, and once you perform an "analogy " of "X - Y + Z = W" its almost certain that W does not exactly match anything in your vector space (probability of overlap is almost certainly zero). That means you must have an algorithm for choosing which non-matching word to pick. The point of the article is that if you pick strictly the closest one (in terms of euclidean distance in the space) then you usually end up back where you started. There are many measures that could be used, and who is to say that distance along all dimensions are equally important?

In 3d space for example (much simplified from 300d), things that are separated along a vertical axis are usually very different in kind than things separated even by a great distance on the horizontal axis (100m below you is likely to be much different from 100m in front of you).

Or imagine a sky scraper most of the "sameness" lies in a small horizontal region (one block say), but a vast vertical region (100 stories).

This is a simplified analogy to word vectors, but the point is that because two words are "close" in 300d space, if we don't understand what those dimensions mean, we can't say which one is more likely to be "similar" for a specific pair of words (King/Queen vs Mouse/Cat). Using euclidean or cosine similarity may or may not be relevant for one particular case.

Der_Einzige4• 7 years ago

Has the author made sure he's using the exact same set of word embeddings as utilized by the authors of that paper? Are they trained on exactly the same corpus with the same parameters? Do the md5 hashes match? This could be cheating... Or it could be a case of model mismatch. I'm left without a clear reason to prefer one explanation over the other.

habitue• 7 years ago

If the embedding relationships aren't stable over different trainings, then they aren't really meaningful right? It's just noise in that case

ajuc2• 7 years ago

BTW sometimes in history that equation was true. For example in medieval Poland Jadwiga was formally elected a king (król) not a queen (królowa). Only several years later her husband became a king and she got demoted to a queen.

konz• 7 years ago

Related submission from 42 days ago: Differences between the word2vec paper and its implementation https://news.ycombinator.com/item?id=20089515

ajuc2• 7 years ago

Fixing "he - doctor" : "she - doctor" to "he - doctor" : "she - nurse"

is kinda sexist.

logfromblammo• 7 years ago

If the system returns the same result word that was in the query, doesn't that just mean that the word doesn't have the intrinsic meanings you thought it did?

Besides that it's actually "King - Man + Woman = Queen [Regnant]", which broken down into smaller pieces should be "King - Man = Monarch", "Monarch + Woman = Queen [Regnant]". Then you also have the complicating factor of "Monarch + Wife = Queen [Consort]" and "Monarch + Husband = Prince [Consort]". It seems obvious from this, and "Husband - Man = Spouse" and "Wife - Woman = Spouse" that "Monarch + Spouse = Consort".

This allows a little thought experiment. If the king marries a man, you have a king and a prince. That's fine; no ambiguity there. If the queen [regnant] marries a woman, you have a queen and a queen. This is confusingly ambiguous. Do you rename the queen [regnant] to king, or do you rename the queen [consort] to princess [consort]?

peapicker• 7 years ago

The whole cartoon about "martini - gin + whiskey" cartoon isn't great, as "cocktail" really is the best answer it could give. (A manhattan would also have to switch from dry vermouth to sweet vermouth, and change the garnish from an olive or lemon peel to a cherry, and traditionally it would not use just any whiskey, but use rye specifically.)

dllthomas3• 7 years ago

A martini can be sweet vermouth (in its early days that was actually more common), but your point overall is valid.

crazygringo• 7 years ago

But then from what I learned it’s not a Martini, it’s a Martinez! And yes it did come first. :)

https://en.m.wikipedia.org/wiki/Martinez_(cocktail)

dang• 7 years ago

A related thread from 2017: https://news.ycombinator.com/item?id=13346104

King – Man + Woman = King?

Related Stories