Gender and verbs across 100K stories: a tidy analysis

http://varianceexplained.org/r/tidytext-gender-plots/

by var_explained • 9 years ago

159 144 9 years ago

stefanwlb• 9 years ago

The author failed to mention the context of the data chosen; specifically the dates. For example, rob vs. steal, would change depending on the century the work was created.

swf3• 9 years ago

The author mentions in the second paragraph that the data comes from scraping Wikipedia articles' plot descriptions. So the plots might be old, but the descriptions (and language) were all written recently.

SerLava• 9 years ago

Hm. Wouldn't that have little bearing on the result? Can't really say "he poisons" when that wasn't the plot of the story.

ghaff3• 9 years ago

Before drawing any strong conclusions, I'd probably want to do at least some validation against original sources, e.g. Project Gutenberg. You're talking about plot descriptions written by a mostly fairly narrow demographic. I'd be hesitant to use that to draw conclusions about the source material.

OJFord3• 9 years ago

For example, rob vs. steal, would change depending on the century the work was created.

Eh?

Both come so naturally to me that I honestly have no idea which centuries you're talking about, or which you think is obsolete.

ghaff3• 9 years ago

Steal has become relatively more popular over time although the difference is probably not large enough to attach a lot of significance to it.

https://books.google.com/ngrams/graph?content=rob%2C+steal&y...

If I had to guess why (other than reasons...) I'd probably speculate that steal tends to have a broader meaning while rob is more likely to be applied to physically robbing someone.

_nalply• 9 years ago

Direct link to a scatter plot (x axis: quantity, y axis: gender):

http://varianceexplained.org/figs/2017-04-27-tidytext-gender...

empath752• 9 years ago

It's interesting how thoroughly ingrained sexist concepts are in the language. Even a verb that's fairly active like 'resist' assumes a power relation in which they are in a worse position.

I'd like to see this done by country or year or language or genre.

oblio• 9 years ago

Heh. You have no idea. English doesn't even have proper genders. You're on easy mode.

If I say: "Your manager is good.", it's neutral. You can't figure out if I'm talking about a man or a woman without something additional.

In Romanian, and I guess in most Latin languages, no such luck. Except for a handful of neologisms, everything clearly mentions the gender, for better of for worse.

The direct translation in Romanian: "Managerul tău e bun" is masculine (both the noun and the adjective make that obvious).

The whole neutral gender movement in English sounds kind of funny for us... we could try to do it Romanian but it would be like lobotomizing the language.

vacri• 9 years ago

A Bulgarian colleague of mine said that even surnames are gendered - her surname is Nakova, her father's is Nakov. It plays merry hell with admin software from anglo nations which assume surnames aren't gendered, apparently.

finkston• 9 years ago

Well, we're talking about gender here, not sex, and given that there are two genders [commonly represented in literature], it stands to reason that they would have different tendencies of behavior, or else there would be no difference between the genders, and the concept would not exist.

bbctol2• 9 years ago

Right, of course. Since we live in a universe devoid of power relationships, it stands to reason that two groups described differently are in fact different, since no one has an interest in describing people other than accurately. Like how the Jews used to be all scheming and evil, but really cleaned up their act in the past few decades.

artursapek• 9 years ago

Is it sexist if it's reflective of reality? Men commit more aggressive crimes than women by far.

shouldbworking2• 9 years ago

Shhhhhh! Theres no room for reality in here, were pretending that gender is a made up sociological concept

zebrafish2• 9 years ago

It's probably still sexist. I think you would probably agree that literature spoke the same way about black people a couple hundred years ago. Cultures can be sexist or racist even if everyone subscribes to the same discriminatory standards.

However, I think the interesting point is that this is the culmination of 100,000 plot descriptions. At that point, it's not a reflection of bias in an author, it's a culturally accepted position on the roles of genders. (whether right or wrong)

...or I guess it could be a bunch of biased wikipedia editors :D

viraptor3• 9 years ago

An interesting question is: does it just reflect reality, or does it shape reality? There are cultures with different ideas about genders - ideas taught to people. So a random book, movie, story, etc you discover in modern west has a higher chance of describing a man murdering a resisting woman. If you see that trope repeated enough times - are you affected in any way? Is there a statistically relevant impact on society as a whole?

I'd say that if there is, then yes, content which keeps the idea alive is sexist to an extent. (Unless you're writing non-fiction) And the existence of the adtech, and ideas like brand awareness, and catchy jingles, and recent fake news is a decent example that society is relatively easy to influence just by constant repeating and enforcing of information.

Torwald• 9 years ago

Let's say your second hypothesis is true.

What would that mean? That men commit more aggressive crimes or that the users of the language perceive it that way? Also, what would then be seen as a crime or as violence?

The point I am trying to make is that there is probably a cultural factor in this as well. (How the culture perceives things to be.)

appetizer5• 9 years ago

But there is plenty of evidence to suggest that isn't an absolute. Men's efficacy at aggression is greater, when it happens, and that is what gets reported.

I would say most of the research in this area is in abusive relationships. Where only heterosexual abuse from man to woman is noticed and acted on, which greatly distorts reality and cultural expectations.

Many men are physically abused by women, and this is underreported.

Gay and lesbian relationships also produce a power dynamic, which has also gone underreported, as they were already either marginalized, or collectively too busy trying to make all LGBT relationships seem completely benign so that their neighbors wouldn't marginalize them.

Do I have enough quantitative evidence to disprove your "by far" statement? No, but thats not the point. There is something about HUMANS that we should enlighten ourselves about.

Red_Tarsius2• 9 years ago

Once again, US culture cherry-picks data and declares that any statistical discrepancy is caused by oppression or sexism. Other hypotheses are taboo and dismissed. You, as well as the author, jump to the morally accepted conclusion without looking at the real world.

I believe in gender equality. But it is worth thinking about other hypotheses even if you don't accept them, that is, the implications of biology in human behaviour.

circlefavshape• 9 years ago

People in literature don't have a biological sex, they only have a socially-constructed gender, and so naturally they will have the characteristics of that gender ... in fact, the sum of behaviour characteristics of the genders in our stories is the social construction of gender

(I started this post off as tongue-in-cheek, but now I realise it's probably true)

Spooky23• 9 years ago

That's a weird conclusion to draw.

Much of this reflects reality. Murder in general is committed more by men. Most reported battery and assault (beat) is committed by men. Murder by poisoning is almost exclusively committed by women. First responders or soldiers who are "saving" people trend male, etc.

When you're looking at 100k fiction works, many of them are going to be romance, mystery and other dramatic type stories that reinforce these concepts.

openasocket2• 9 years ago

The reference at the end talking about comparing changes over time reminded me of a problem I've been kicking around. For this particular system that's not too hard, since you're putting each word on a one-dimensional femininity/masculinity scale, so you could plot a word on a line graph or something. But what do you do if you want to evaluate the changes of more complex relationships over time? Not just mapping words to a constant vector space, but modeling the relationships between words, such as clusters or word2vec representations. With something like word2vec you can take a bunch of words and project the vector space onto a plane so you can see the relative distances, but how do you express changes over time? You could show a bunch of planar projections for different instants in time, but it's hard to look at that and capture the changes.

So how do you visualize changes to these more complex interactions between data points, and also how do you mathematically quantify some of these changes? I'd really appreciate any advice on this. And sorry that this is kind of off topic for the article :)

joshuacc• 9 years ago

With something like word2vec you can take a bunch of words and project the vector space onto a plane so you can see the relative distances, but how do you express changes over time?

Animate it?

zebrafish2• 9 years ago

I don't think I understand what your end goal is.

what do you do if you want to evaluate the changes of more complex relationships over time?

Which complex relationships are you talking about here?

Edit: I think I understand what you're asking. Maybe one good way of doing this would be to cluster word groups for different periods of time. You could then perhaps look at where a word is now, where it was at the beginning of "time" and take the delta to examine words whose meaning has changed the most.

Another approach might be to do apriori over time. Take a text from 1600 and lhs is "some manly action" and rhs is "he". Take a text from 2000 and lhs is "some formerly manly action" and rhs is still "he" but with much lower levels of support.

datalink• 9 years ago

One way is to learn clusters in the data over the whole time period and then calculate the cluster distribution of the same clusters for each time interval separately. You can then track the proportion of each cluster over time as a time series.

So for example if you have a word cluster that describes computers, you could see it start growing in the seventies, while having near-zero proportion in 19th century etc.

jasonjayr• 9 years ago

The Gapminder charts very nicely illustrates multi dimensional data over time:

http://www.gapminder.org/tools/#_locale_id=en;&chart-type=bu...

I imagine a dataset that charted the word use over time would fit into a chart like this.

viraptor3• 9 years ago

What about 1d protection and simple line plots over time? You'll need 2 or 3 of them over some interesting projections, but it's better than unfamiliar and complex representation.

Alternatively 1d protection + style change for another axis. Like changing the colour, thickness of the line, etc.

swf3• 9 years ago

I did something like this in a project for my former work at a research group, and it was very difficult to visualize due to the number of variables we needed to communicate in the visualization.

What we were trying to do was use word2vec to model changes in relationships between 700 proteins and one in particular, which is related to cancer/tumor growth. We created multiple word2vec models based on year windows of medical journals (so the 2003 model had input journals from 2000 - 2003).

To visualize the models we used a D3 force graph where the nodes were the 700 proteins and the edges were known discoveries of relationships between proteins that had years associated with them - as in X protein was discovered to be related to Y protein in 2007. The relationship data was curated by people, independently of the word2vec models. The size of each of the nodes was determined by the word2vec model for the particular year's similarity score between that protein and the cancer-related protein we were interested in.

To see the changes between years, we used a year slider which the force graph would respond to by animating the sizes of the nodes in line with the particular year model's similarity score for each protein. In addition, the color of the nodes represented changes in the models' similarities between the proteins - more green meant that the word2vec model's similarity score had increased compared to the previous year's model, and red meant it had decreased.

The visualization is useful, but it is a bit of a mess considering there are 700 . I'll message you the link and if anyone else want the link I can send it, but I'd rather not post it here since it's hosted on my college's CS department server and it's not equipped to deal with a lot of traffic.

Also if anyone has an idea of how else we might have done it I'd be interested to hear it.

Edit: Didn't realize HN doesn't have a PM feature - if there's another way to send it to you and you're interested let me know.

openasocket2• 9 years ago

Thanks, that's helpful! And you can email me at <my HN username> at gmail.com.

I'm also curious if there are any ways to quantify, mathematically, the changes over time. There's the simple sum of the squares of the changes distance to get a sense of the "kinetic energy" of the system, but I'm wondering if there are some more clever analyses, especially something that can quantify localized changes versus global changes.

Edit: so are you running a separate word2vec thing for each year's dataset? If so, how to you map between them, because the orientation the word2vec mapping generates will be random, and I worry that trying to rotate the mappings to some common axis could obscure some of the data.

jameskegel• 9 years ago

HN : Social Issues :: general conversation : a vegan

Neliquat• 9 years ago

That cuts so many ways... But more than ever, I see a great diversity of social outlooks on HN. Might be recent politics, but the result is the same. I love the sub arguments on lab meat, tech ethics, gender equality and representation, and vi vs emacs. Even when I do not agree, I find coherent and consistent alternate points of view useful in seeing the world.

wyldfire2• 9 years ago

what verbs are used after “he” and “she”, and therefore what roles male and female characters tend to have within stories.

It might've been better had the author (and the Jane Austen article's author) used some NLP processing to see whether the pronoun was actually the subject of those verbs. But I'll grant that it's usually the case.

Also interesting: gender and the object of those verbs.

EDIT: after some research (that I should've done before posting), it's a remarkably effective technique and it seems only the most contorted sentences might get tripped up. English is nearly always Subject-Verb-Object.

burkaman• 9 years ago

Can you give an example sentence where this isn't the case?

dahart5• 9 years ago

Do you need NLP for that?

The forms he, she and they are used when a pronoun is the subject of a sentence. The forms him, her and them are used when a pronoun is the object of a sentence.

https://www.englishgrammar.org/words-heshe-himher-hishers/

I would think looking at the verb that comes after he/she also doubles as a subject/object filter.

coroxout• 9 years ago

I agree that the nominative case endings and usual SVO order of English should prevent (nearly all) false positives.

(The only one I can think of is that comparatives are meant to take the nominative case, e.g. pedantically it's supposed to be "he" rather than "him" in "A man smarter than he would decline". However, this rather convoluted sentence structure is rare, even more unlikely in a plot summary rather than the full novel text, and almost always followed by the subjunctive rather than an indicative verb.)

My bigger concern was what doesn't get caught because it appears after a noun rather than a pronoun, or has an adverb in the way. I thought major dramatic plot points might be more likely to use the character's name rather than a pronoun, and so we might see fewer words like "murders", "defeats", etc. From the results it seems like those words are still present in large quantities, so perhaps I'm wrong.

I'd like to see it done with "they" too, partly as a control case and partly to see if any verbs are more common for individuals rather than groups (although the rise of impersonal "they" may hinder that aspect of the analysis).

jayd163• 9 years ago

One thing that might cause error is the difference in pronouns vs proper names names or other labels.

In the example "The man winced, then he fell to the floor." You'd only register "he fell." This would also be biased if male or female characters were named more often.

merricksb• 9 years ago

Gender and verbs across 100K stories: a tidy analysis

Related Stories