Overwatch Randomized Trial on Gender

by Aissen • 7 years ago

202 109 7 years ago

jasonshen• 7 years ago

This is about as well done of a study as we could ask for from a non-academic research level. Basically while the number of sexual comments were relatively low, the amount of questioning/challenging of his abilities went up a lot when he played with a feminine name.

Reminds me of a professor at Stanford who underwent a sex change and experienced very different interactions from his peers: https://www.nature.com/scitable/content/does-gender-matter-b...

We recently did a survey of 240 collegiate esports players and found that women were 9x more likely to be "very concerned" about sexism and 2x as likely to be "very concerned" about bullying in competitive gaming culture.

I think Dan's research can help explain why.

https://www.midgame.gg/survey

towaway11382• 7 years ago

about as well done of a study as we could ask for from a non-academic research level

Unfortunately, there's a good reason why academics use proper experimental design. Without it, all you really have is suggestive noise, if even that.

macleginn• 7 years ago

The definition of “proper experimental design” varies widely by the field, and DL’s methodology looks quite solid, as far as social sciences and the task at hand are concerned.

Nexxxeh3• 7 years ago

Really interesting read. I appreciate that it highlights the problems with the data and doesn't reach ridiculous conclusions based on junk data.

Anecdotally I've found a noticeable improvement since Endorsements, which this data missed.

Also the rise of Fortnite and more recently Apex Legends has led to a further decrease in toxicity that I've seen. A lot of toxic kids left for BR pastures.

That's on PC in the EU region on a high-silver current, low-gold peak SR account, casual QP-heavy play but some comp, pure flexing (as I prefer to fill rather than focus on maining one or two characters. Maining is arguably the best way to climb out to Plat).

I'm an almost ridiculously positive player with a gender neutral name and I've found being overtly positive vastly improves the experience.

It does prompt a couple of toxic responses ("kys", "fuck off" etc) but the majority is really positive. (I generally just report any toxicity.)

Playing without voice in Comp is sub-optimal, as winning is much easier with some kind of synergy. Even a basic PharMercy combo, or ultimate combos like a EMP+DVa Bomb can be highly effective.

I normally Solo Q, but playing paired with a friend in QP yesterday I had a surreal experience. One player on the enemy team insulted my friend (sexual but not specifically-gendered insult). She was playing with a non-gendered name, as Moira, and the toxic enemy player complained she was carrying us.

All the players on both teams turned on the toxic player, calling them out and highlighting why they were actually losing. I have not seen anything like it before but it was hilarious.

Stay As A Team is great in theory, but a team of 6 randomised will get rekt by a pre-made 6 stack on Discord with each other, and the matchmaker doesn't seem to take that into account.

I wonder what region author was playing on?

kfwhp4• 7 years ago

Every time I see a feminine name in a videogame I think the person is just fishing for attention. This is important because most players of both genders use neutral-looking names. The study doesn't account for this.

SolaceQuantum3• 7 years ago

Honestly, I don't really see such behavior directed towards male names though. Stuff like LucasJones or MikeAtLarge or LordOfKilling doesn't get the same attention that SuzieHealsU or CuteLamb or PrincessHurtful might. If it was equal, I'd expect 'MrKickass' to be questioned and assumed to be grabbing attntion that 'MsKilljoy' gets... but in my experience this distinctly isn't the case.

ggggtez4• 7 years ago

The thing you are saying doesn't even make sense as a criticism of the study. You are complaining about something the study didn't even try to measure (the "why"). The study only tried to measure if there was a difference at all, and whether it was measurable.

As a side note, I recommend meditating on why seeing a female name on the internet upsets you, when you should be well aware that 50% of human beings are female. If players didn't get upset and force females to play under male/neutral names, maybe it wouldn't be as shocking to notice when they don't.

Vector919• 7 years ago

This is important because most players of both genders use neutral-looking names. The study doesn't account for this.

Is this true broadly? I'd be curious to know if there's any data on that besides for anecdotal evidence.

also, a "neutral-looking" name can be really subjective, I would guess that there's a lot of player names that use adjectives that might be inadvertently related to a particular gender based on the cultural context.

I'd be curious to know what the names are in this case.

gbrown2• 7 years ago

Why should a gendered name invite more attention in any way? People are often given gendered names at birth, after all.

ben_jones2• 7 years ago

Overwatch remains one of the most toxic games I've ever played. I've put well over 1,000 hours into that game but haven't played for about 9 months now, and honestly its a breath of fresh air (to not deal with it anymore). There's something about the rank structure of the game (and others I've heard about like LoL and Dota) that encourages beratement.

Apex Legends conversely has been amazing in comparison. Maybe its because its new, the tagging system, that its a free game, or that there is no ranked structure (yet), but I've met nothing but cheery people and the occasional crude drunk. I've played with a couple shy women, but (anecdotally) have seen not one iota of what i've seen in Overwatch (aside from crude usernames that are prevalent everywhere).

kfwhp4• 7 years ago

that its a free game

Wait... Shouldn't it be the opposite? Paying for the game should make you more reticent to being an ass because losing access to it means you've lost money.

woolvalley• 7 years ago

When I played overwatch in a casual way a few years ago (never ranked) I definitely rarely encountered any sort of toxicity. It was much better than dota for sure.

Dota has gotten better as a game as years gone by.

tetraca2• 7 years ago

It's likely because there is no ranking structure. DotA 2 used to have nicer community, until rankings were implemented, at which point the number of people who explode at the slightest error very noticeably blew up. Once you add in rankings, not only do people get mad at any possible chance of losing rank points in ranked mode, unranked mode becomes seen as a playground where a certain contingency of people think it's okay not to try at all. As a result the game deteriorates in quality no matter which path you pick.

jdietrich3• 7 years ago

Overwatch remains one of the most toxic games I've ever played.

Try solo queue comp CS:GO. CYKA BLYAT, IDI NAHUI PIZDEC.

crummy• 7 years ago

Agreed on Apex Legends, which now that I think about it seems kind of odd. I think a game that builds tension like that, where games can last 20-30mins, where a teammates mistakes can easily cost you the game, is asking for toxicity, but I don't recall any at all in the ~20ish hours I've played so far. And that's with a lot of mic usage (both me and other players).

Nexxxeh3• 7 years ago

My first game of AL was great, and I was paired with a talkative and helpful teammate and one silent but cooperative one. After that it ranged between silence (Comms and text) to outright hostile. I like the tagging system and appreciate not everyone can voice Comms, but everyone can type hi. I went back to OW.

Not to say OW is without problems, obviously. Not related to OP is throwing and smurfing. I was in a Stay As Team group that got dumped into a large group of smurfs. They let us cap and win and didn't even contest, but would occasionally just mess with us to show off their mechanical skill. Most of them could have 1v6'd us. Most of our team reported them, but it was just really disappointing.

OW definitely has problems, I just find them less than other games.

What region were you playing in, and have you tried others? I get that ping might be an issue, speed of light being uncooperative and all, but I'd be curious to know your experience based on region and what times you play.

baud147258• 7 years ago

I wonder what region author was playing on?

I think it's North America.

ggggtez4• 7 years ago

Obviously, a number of issues present in the study, but I can appreciate they tried to account for some of them. Lots of variables are unaccounted for like time of day, win/loss, character, what the researcher said in chat, etc.

Similarly, the opening preamble points out that often the worst harassment is potentially related to when players hear female voices. Obviously there is a different way you could do this study, which would be to invite ~300 players of various skill levels to play a single game and record the results for analysis. That type of study would have it's own problems, but you could at least be more likely to control for some of the problems in this study.

mactrey• 7 years ago

Yeah, the voice is the key. The gender ratio is skewed so much at higher ranks that an obviously feminine name will simply be assumed to be ironic. I would imagine the toxicity/sexual comments would increase by an order of magnitude if the voice was female-presenting along with the account name.

novia• 7 years ago

Or this researcher could have got a woman to speak into his mic while he played.

AcerbicZero• 7 years ago

This is an interesting study, and helps highlight why Overwatch needs a scoreboard. With 6 random people on a team, very limited personal statistics, and no game length team statistics, everyone tends to get toxic once you start losing. I've played everything from high gold (tanked rank for a few seasons) to low masters, and its the same pattern every time. Someone makes an accusation about a player not pulling their weight, demands a swap in a rude way, and the player being accused defends themselves. Then the name calling starts.

It would be interesting to see what roles the author was playing, and the exact SR of the accounts, as there seems to be a great deal of variation in what people will say on voice comms (usually higher SR, because voice comms > chat) and what will be said in chat. People also like to blame DPS first, then demand a Reinhardt (if they don't have one) and then finally blame support.

Without hard data as to what is going wrong with a team composition, toxicity is almost guaranteed when losing.

quxbar• 7 years ago

To be fair to the paper, toxicity is almost guaranteed when losing while playing with a feminine name. It's pretty rare otherwise.

wlesieutre• 7 years ago

People also like to blame DPS first, then demand a Reinhardt (if they don't have one) and then finally blame support.

As someone who mains support, I don't think this is true. I've had plenty of games where I do a shitton of healing, get the healing card at the end, and still had people blaming me for "0 heals" all game.

A scoreboard would help with that, but it'd be a trade-off with having 5 angry people shitting on you if you ever have a bad game. I'm not sure that's better.

EDIT: Plus there's all the times when your stats are bad but you're not the right one to blame, like when the whole team can't move up through a choke because the tanks are too chicken to go first, but it gets reflected in poor DPS eliminations.

Ultimately it's a team game, and failures of teamwork aren't reflected well in a stats scoreboard. I think you'd incentivize poor team play by people trying to pad their own numbers at the team's expense.

And no matter what information is available, some people are just looking for someone to blame when they lose. I'm not sure how to fix that. If matchmaking is working right you're going to lose about half the time, there's no point in getting worked up about it but people still do.

greedo• 7 years ago

Exactly. I average around 10k healing/10min, but if the team isn't working well, that can drop 90%. If no one peels for me, or if we just get rolled because of an infinite number of variables, my individual stats can look like crap.

jdietrich3• 7 years ago

I'm not entirely sure that a scoreboard would help. I play a lot of CS:GO matches where people are playing for the scoreboard rather than playing for the team. Statistics can never perfectly capture the contribution a player makes.

Scoreboards can also give people license to be toxic. If you're on the losing team in CS:GO, it's pretty common for the top fragger to start cussing out the rest of the team as noobs, even if that player is greedily hunting frags and the other players are making a useful tactical contribution.

cloverich2• 7 years ago

there are so many nuances I’m skeptical there are meaningful metrics that would help. I’d honestly prefer no medals or scoreboard leaving just the win or loss. I’ve never seen anything constructive come out of it.

NullPrefix• 7 years ago

There are a lot of games that weren't included in the experiment because I wasn't in a mood to listen to someone rage at their team for fifteen minutes and the procedure I used involved pre-committing to not muting people who do that.

SolaceQuantum3• 7 years ago

"If that result generalizes to OW, that would explain something which I thought was odd, which was that a lot of demands to switch and general vitriol came during my best performances with the feminine account. "

This isn't statistically analyzed, but I think it's a fairly significant detail that's not emphasized more in the paper.

qiqing• 7 years ago

Agreed. Especially this excerpt:

Per above, I didn't want to code tone of messages to avoid bias, so this table only indicates the rate at which people told me I was playing incorrectly or asked that I switch to a different character. The qualitative difference in experience is understated by this table. For example, the one time someone asked me to switch characters in the masculine condition, the request was a one sentence, polite, request ("hey, we're dying too quickly, could we switch [from the standard one primary healer / one off healer setup] to double primary healer or switch our tank to [a tank that can block more damage]?"). When using the feminine name, a typical case would involve 1-4 people calling me human garbage for most of the game and consoling themselves with the idea that the entire reason our team is losing is that I won't change characters.

nickpsecurity• 7 years ago

Great study! Only gripe is that most trolling I get in online gaming is through voice chat. They also seem to troll me a tad bit more if I have a mic plugged in but I'm not talking. I suggest a re-run of this experiment recording the audio in case we get wildly different results. In a one-party consent state if you want to play it safe. ;)

I also think it will vary game by game. I don't play a lot of games. Most are shooters like Modern Warfare, Battlefield, and Mass Effect 3. Modern Warfare 2 and 3 on Xbox brought in more trolls in voice chat than all my others combined except maybe GTA V. In the last one, you play as a vicious thug causing problems for everyone around you. Them being pricks makes more sense. The difference in shooter games, though, was more interesting given it might reflect different sub-cultures in gaming. Specifically, MW is designed to appeal to largest number of people with lowest barrier to entry and patience required. Might self-select for higher percentage of jerks. Whereas, Battlefield 4 and Mass Effect 3 push people to work together more for best results.

towaway11382• 7 years ago

I don't understand why people ever play with voice chat enabled. I don't want to hear streams of ungrammatical foul language, and I certainly don't want to hear people slapping their kids, etc. TMI.

jdietrich3• 7 years ago

It's absolutely necessary if you want to perform well in a team-based game.

dragontamer4• 7 years ago

Any team game has far more efficient communication when you have voice chat.

Heck, even a "casual" MMO like PvE Guild Wars 2 games, have chat-rooms and voice-chats because its way easier to specifically say "My Rocket Boots are charged, can anyone drop a light circle?". And that's not even truly competitive or "hard" compared to FPS games.

Go into PvP Guild Wars 2, and you won't get anywhere without voice chat. Coordination is a huge element of the game.

munchbunny• 7 years ago

I've found that when you are matched with a team of good communicators, voice chat is a force multiplier. I've had games of Overwatch, Battlefield, etc. where the faster coordination really let us steamroll the other team.

Maybe the other players were just really good and I was getting carried? But from my anecdotal experience, good voice chat tends to add a ton to team momentum.

With someone who's playing TV in the background, someone who doesn't bother with callouts, and someone who's tilting hard... I'd rather mute them and myself.

antisthenes• 7 years ago

Perhaps I missed it in the article, but did the author control for receiving comments on 'how to play' in winning games vs losing games?

Because if not, the whole study seems sort of moot.

tdb7893• 7 years ago

The hardest thing about this study to me is that these tendencies aren't only applicable to Overwatch. It's obviously worse in video game culture (because it's more abusive and immature as a whole) but this matches with what I've had women engineers tell me of their experiences (especially in college where I know people who got asked out constantly and also were disrespected in group projects)

zemo2• 7 years ago

The entire premise of this study is not useful, because what you're asking is "what is the numerical occurrence of quantifiable harassment" and not "what is the rate and which players abandon playing the game as a consequence of feeling unwelcome and how does it vary by gender". What do the numerical rates of harassment tell us, and how are they actionable? If a group of players does not play because they feel unwelcome, THAT is the thing that matters, and your study does absolutely nothing to analyse this criteria.

I avoided ever using voice chat and would call things out in text chat when time permitted.

Again, this betrays a deep flaw in methodology, since voice chat is an integral part of competitive Overwatch. The important things to consider here are: what percentage of players are comfortable using voice chat and how does that vary by gender; does availability and usage of voice chat confer any competitive advantage to the player; and do the two factors interact in such a way as to confer a competitive advantage to one gender over the other?

I find it disturbing when men do scientific studies to determine if women's stated experiences are real.

minitech2• 7 years ago

If a group of players does not play because they feel unwelcome, THAT is the thing that matters

What, in terms of profit? (And in the next paragraph, competitive advantage, really?) That’s a very narrow view of what it means for something to “matter”.

zemo2• 7 years ago

no, in terms of the fact that they literally ran away because of harassment. That's important, and if I have to explain to you why people leaving due to serial harassment is bad for communities, we have nothing to talk about.

hexane360• 7 years ago

Why can't we have both? The response to studies of the nature you suggest is often "maybe women are more sensitive" or "facts don't care about feelings" or "everyone gets harassment".

leetcrew• 7 years ago

I find it disturbing when men do scientific studies to determine if women's stated experiences are real.

women as a group do not have "stated experiences". individual women do, and if you talk to enough of them you'll find out that they have very different experiences and opinions.

most women I have talked to who play competitive video games do say that they experience gendered harrassment, and I don't see any reason to disbelieve their individual accounts. on the other hand, I don't actually know very many women who play these games, so my sample is pretty small and possibly skewed by all the filters that influence whom I am likely to have a conversation with.

if I cared a lot about this specific issue and wanted to collect more data and apply stastical techniques, why would that be disturbing?

dev_dull• 7 years ago

For one thing, when I was playing video games in the 90s, a huge fraction of the online gaming population was made up of kids, and online game communities were nicer than they are today

Maybe it’s because I grew up playing FPS, but for as far as I remember people in these games were extremely hostile to women.

ntsplnkv2• 7 years ago

Nostalgia is a powerful thing.

I will say, though, dedicated servers which were far more common back then typically had admins that would moderate servers. On good servers toxicity was almost nonexistent. I miss those days a lot.

I like matchmaking of today a lot too, but some of my best memories came from the communities built on dedicated servers.

aidenn02• 7 years ago

I think community expectations are important. People in general, (maybe kids especially?) want to fit in, so they will meet whatever the perceived expectations of their peers are.

dragontamer4• 7 years ago

I mean, I grew up in the 90s playing "Neopets" and I remember that the online gaming population was made up of kids, but overall more friendly.

I also played Starsiege Tribes and Starcraft Brood War online, and... yeah, a bit more toxic over there. The "hardcore competitive" community just oozes toxicity for some reason as far as i remember.

But even "Yahoo Chess" (a very peaceful community) would have a difference between my Sister's chat and my chat when we were kids. People really do treat women differently online. My sister was slightly better than me in Chess (she probably won 60% of the time vs me), but otherwise we both were roughly on the same level. The only major difference would be our usernames.

crazypyro• 7 years ago

Competition can often breed toxicity. I've actually found Starcraft to be less toxic simply because in the most competitive game mode ( 1 versus 1), there's nobody to blame but yourself if you lose. People tend to get toxic about using "cheesy" strategies or playing the "overpowered" race, but overall, the most toxic communities I've been apart of tend to be highly competitive games where team play is a key component.

Some examples in my experience are games like Counter-Strike, Dota 2, LoL, etc. A couple common features of these games are:

1. Matches tend to be drawn out over at least 30 minutes, if not longer, forcing you to be teamed with people you may not get along with for a longer period.

(Overwatch comes close to having this issue, but I still feel the shorter match times tend to limit extreme toxicity you can find in other games)

2. A single player actively working to harm the team, or simply not at the same skill level as the rest of the players, can easily sink the team's chances of success.

3. An individual matchmaking / visible ranking system which fosters toxic attitudes like "well, I'll sink your rating if I don't like you", since every member of the team is ostensibly trying to raise their rank.

#3 when combined with #1 and #2 can create situations where "trolls" can easily ruin a game that may take an hour to finish and may even prevent any chance of victory.

These factors tend to be key to the most toxic game communities I've been involved in.

superkuh• 7 years ago

Just as a tangent, Starsiege: Tribes is still around and active on the friday+weekends. The community took over patching a decade ago and run custom coded master servers now. You can get a modern client at https://playt1.com/

20 years on and it's still going because of community modding and dedicated servers. I'll eat my hat if Overwatch is available in 20 years.

aidenn02• 7 years ago

My very fuzzy recollection makes me remember that DWANGO and Quakeworld gamers were more hostile to women than subspace games, and Dan mentions in the article that he played Subspace.

slavik812• 7 years ago

Overwatch is the only First-Person Shooter I have ever played where I have heard women speak in voice chat and other players continued to focus on the game itself. In other games I've played, a woman in voice chat would always become the center of attention, whether she wanted to be or not. The attention wasn't always negative, but it was unavoidable (and thus probably pretty frustrating).

That happened in Overwatch sometimes too, but I was surprised it wasn't more often. My suspicion is that it's the a result of the competitive mode putting focus on the gameplay and a larger than average female player base for an FPS.

danso2• 7 years ago

Came here just to say this. Before OW, my other multiplayer games were Valve's (CS, TF2, L4D), and I didn't know any girls/women in real-life who played them. Given the hours/years of time I played, I would have to guess that I played in a game with a female player at least a few times, but I could not for the life of me ever recall a single time. It would be such a rare occurrence that if it did happen, I think I would remember it generally.

With OW, I'll hear a female voice at least a third or fourth of my random games, as well as see female-gendered screennames who may not actively participate in chat. I haven't played many games between old-school CS and TF2, so I don't know if the overall demographic has changed. But I wouldn't be surprised if OW's diverse and welcoming cast and lore was a major factor in attracting female gamers. As much as I liked the older games, OW was the first game that made me realize how much diversity could itself be a feature -- not just in attracting a playerbase, but just more creative designs (though I understand it may be too cartoony for some, of course). If only Blizzard's writing could be as clever and less clichéd-ridden than Valve's...

twic• 7 years ago

I've experienced that plenty of times in Team Fortress 2. There are not many women playing it and using voice chat, but when they are, in my experience, it is vanishingly rare for it to be a big deal. FWIW, i'm in the UK, and so mostly playing on European servers. I suspect that relative to other FPSs, non-competitive TF2 players skew older and less tryhard, which may explain it.

devindotcom• 7 years ago

Examples of typical undirected comments that would occur in either condition include ""my girlfriend keeps sexting me how do I get her to stop?", "going in balls deep", "what a surprise. strokes dick [during the post-game highlight]", and "support your local boobies".

I think it's notable that although these are not directed they are highly gendered. This is part of the problem as well.

macrael2• 7 years ago

This video series from a gamer who plays with voice chat enabled was pretty eye opening. https://www.youtube.com/watch?v=jL0aVqVslSE

34qg4g• 7 years ago

Cherry picked recordings are very eye opening

macrael2• 7 years ago

The series comes out regularly enough to give you all the information you need about the frequency of this shit.

rossdavidh• 7 years ago

So, it seems like you could probably get a fair number of people to volunteer to participate in a study like this, in order to up the number of games played. Also, I wonder if you could (Laurie Anderson-style) change your voice's gender, so that gender could be manipulated even for games with live voice. If there's a researcher out there who was willing/able to organize, I think there would be a fair number of volunteers.

madrox• 7 years ago

I do wonder if toxicity is inherently built in to the nature of Overwatch. It's a 6v6 team game that incentives you into a play mode that punishes you heavily for leaving a game you aren't enjoying. I realize non-competitive modes are a lot less punishing, but anyone who's played a lot of Overwatch knows if you want to "play Overwatch the way it was meant to be played," you have to go to competitive. Speaking psychologically, I'm curious what that kind of experience does to a person's behavior (short term and long term).

I've played hundreds of hours of Overwatch on console with my wife. We don't use mics, because we had enough bad experiences early in our career to make it not worth the risk. I never have problems when I play by myself.

Surprising number of people questioning the methodology in this study. I would've expected that if it ran counter to anecdotal experiences described by women (and men who group with them) who play, but I'm not convinced more rigorous methodologies would significantly change the results. If someone does believe that, I'd seriously challenge their reasoning.

eropple2• 7 years ago

I don't think toxicity is built in, though I understand where the thinking comes from. I play Overwatch (albeit on PC) and pretty actively use a mic, and I make a point of being the first person to talk in a channel. I'm friendly and I'm even-keeled about it, and that sets the tone. Once that tone is set it's very difficult for your garden-variety douchebag to start mouthing off.

The LFG system also helps.

JimboOmega• 7 years ago

Comparing overwatch to League of Legends, League is sooo much worse, and I don't know why.

Also FWIW the voice you use in OW can cause drama. I've definitely had games where I get flak for my voice - when it was more androgynous often people would want to know my gender.

I've seen people get exploded on for their accents, sounding young, etc.

bcassedy• 7 years ago

Yeah, games where someone kicks things off in a positive manner or with a joke almost never turn into toxic nonsense. I wish I were better at being the one to do that.

cloverich2• 7 years ago

I think people are simply very good at scapegoating. I’ve seen so many instances where someone was targeted for something blatantly inaccurate e.g. “rein never used his shield”, then rein gets a card specifically calling out high damage blocked, person continues to scapegoat. The only thing that changes at higher SR is the targeting is somewhat more accurate, but it’s always the same cause: tilting and wanting to blame someone else.

belorn• 7 years ago

To loosely quote Richard Dawkins from the god delusion, the purpose of a theory is to formally describe something in order to question it. Questioning the methodology of a study is to derive understanding, rather than belief which neither need theories nor want questions.

The result did run counter to anecdotal experiences described by women, as is mentioned in the second section. The female account did not see more sexual or sexualized comments, out of the whole 339 tests only one was definitely address to him and a second event that the author is uncertain about. As describe in depth by the author, outliners like those are of questionable value. That is very far from the anecdotal experiences usually described about competitive online team games.

The questioning/challenging of abilities was the other finding, which the author speculate is derived from when a player is underperforming and get frustrated they lash out at team members who is overperforming. This is where a gender difference is found and it is an excellent starting point for further studies. For example it would make for an interesting path to compare studies that look at game theory experiments where player punish generosity, and in cases punish generosity more than defection.

I would also like to see a more in depth study on exactly what underperforming players do when they get frustrated. What is the common outlets and how they differ depending on circumstances. The author did mention that they were more often the overperfoming player when he used the female account so its possible the methodology in this study actually did influenced the result, which the author says in the article.

I found this study interesting but it did not change my view about the issue. I have always though that the difference in online game experience comes from the method people use when trying to insult someone else, rather than the rate in which people do it. If the only clue someone has is the name then that is what people will use in order to guess what method create the best insult, which results in primarily a gender difference.

MobileVet• 7 years ago

Lots of one off gender swaps that have been shared and showed similar bias. Eg https://blog.olark.com/live-chat-gender-equality-experiment

ToFundorNot• 7 years ago

This study is missing one important factor, a baseline. The author should have provided an androgynous named account to see how that fared.

Dylan16807• 7 years ago

An A/B test doesn't need to attempt a "middle" option. And there's no reason to assume an androgynous name would be in the middle or in any way qualify as a "baseline".

zaarn2• 7 years ago

It would still be very interesting to see how people would react to neutral usernames like randomly generated number sequences. It could provide some useful insight.

jordanthoms• 7 years ago

If I took up soccer, I'd be unlikely to join a team with some 8 year olds - even though I'd probably be at their skill level. Instead, I'd find a group of people who I'd get along with better. So why do games do matchmaking based only on skill, not taking social cohesion into account? There's much more to a good gaming experience than just having skill-balanced teams.

CS:GO is a classic case of this - the player base is noticeably more mature at certain times of day, and there's a distinct period when all the kids have just got home from school where things are particularly bad. Occasionally you'll get a team where you everyone works together well, but to have that every time you'll need to spend a lot of effort maintaining a huge friends list in order to be able to put a team together at any given time - rather than just pressing the matchmaking button.

(I think you could might get decent results on this by having each player rate the others at the end of the match - but you don't use the ratings to rank the players, instead you'd use them to cluster players based on who they mutually like to play with. You'd then use that as a variable to optimize on when doing matchmaking. Hopefully with enough data you'd be able to predict if players would have a good experience even if they've never played together before.)

nordsieck• 7 years ago

I think you could might get decent results on this by having each player rate the others at the end of the match - but you don't use the ratings to rank the players, instead you'd use them to cluster players based on who they mutually like to play with.

Dota has a system like this called "behavior score". I know it works based on reports, but it might include other things like text analysis or item feeding. It seems to work pretty well.

AnIdiotOnTheNet• 7 years ago

Since matchmaking has taken over, I basically don't even play online multiplayer games anymore. I feel fortunate to have grown up in a time where there were dedicated server communities. It's a bit like Cheers, sometimes you wanna go where everybody knows your name and you don't really mind that they routinely wipe the floor with you.

Mirioron3• 7 years ago

So why do games do matchmaking based only on skill, not taking social cohesion into account? There's much more to a good gaming experience than just having skill-balanced teams.

Not enough players. Any matchmaking system requires a large number of players to be playing at once to work. People are generally unwilling to wait several minutes or more to get into a game. The problem is that this has a cascading effect: the longer your queue times the more people will quit and the longer your queue gets. Below a certain threshold the system breaks down. If you start classifying people based on criteria such as social cohesion then you increase matchmaking time even further.

There are some games that could have a matchmaking system like this. But no developer knows in the development stage that their game is going to have this kind of popularity, which means that this type of matchmaking would have to be built after the fact. At that point you risk ruining something people like about the game.

Another thing to keep in mind is that player numbers fluctuate significantly. The number of players playing at 5 am between Monday and Tuesday is much lower than the number of players playing at 7 pm on a Friday evening. You would have to change matchmaking algorithms based on time of day or activity.

You would also have to change matchmaking algorithms based on skill level. At the very top end matchmaking is going to be long due to there simply not being enough players at that rating. This can have a negative impact on the retention of players, because popular streamers usually have a high rating. Watching a stream of matchmaking queue isn't very interesting and doesn't make people want to play the game.

Then there's the question of whether you even should do this. People make friends in games. If a game introduces matchmaking on social cohesion you essentially put players into a filter bubble. We see from Facebook and Reddit that these kinds of filter bubbles aren't ideal. I think one of the coolest things that I've got from games is that I've interacted with and made friends with people wildly different from me.

gpm• 7 years ago

Lots of comments here about voice chat.

I wonder if there is software that can convincingly change the gender of a voice in real time? Could make for an interesting experiment.

Overwatch Randomized Trial on Gender

Related Stories