About Google's approach to research publication – Jeff Dean

https://docs.google.com/document/d/1f2kYWDXwhzYnq8ebVtuk9CqQqz7ScqxhSIxeYGrWjK0/preview?pru=AAABdlNwLxs*PKCOHN-Ks0PI5nFrljenMg

by yigitdemirag • 6 years ago

526 573 6 years ago

docs.google.com

gcr3• 6 years ago

Maybe different teams are different, but on my previous team within Google AI, we thought the goal of google's pubapproval process was to ensure that internal company IP (eg. details about datasets, details about google compute infra) does not leak to the public, and maybe to shield Google from liability. Nothing more.

In all of my time at Google AI, I never heard of pubapproval being used for peer review or to critique the scientific rigor of the work. It was never used as a journal, it was an afterthought that folks on my team would usually clear only hours before important deadlines. We like to leave peer review to the conferences/journals' existing process to weed out bad papers; why duplicate that work internally?

I'm disappointed that Jeff has chosen to imply that pubapproval is used to enforce rigour. That is a new use case and not how it has been traditionally used. Pubapproval hasn't been used to silence uncomfortable minority viewpoints until now. If this has changed, it's a very, very new change.

logicslave• 6 years ago

Do you want to know whats interesting? I read alot of computer science research, particularly what comes out of Google. Its clear to me that details are left out of specific papers, especially how things are done in sub systems. But, like a jig saw puzzle, I discovered that many papers are actually descriptions of computing systems and algorithms that interact. If you read between the lines and squint your eyes, you can get a much bigger picture of internal google AI systems than you guys think you can.

SirYandi• 6 years ago

Would love to read a blog post of your observations about that

cbsmith5• 6 years ago

Oh yeah, absolutely. It's not at all hard to see the systems view of this.

vikramkr• 6 years ago

They might have enough of a PR budget to make the Google version of the story stick. But, its concerning that, if what you say is true, they are hoping to make that work by leveraging the public's ignorance of how the Google specific process works. Its also not the smartest move, since Google is important enough and public goodwill towards tech is low enough that journalists will have a field day looking for evidence of double standards/a cover up. And they're not making it too difficult for the journalists if that evidence is found in the top comment on a hacker news post.

hackcasual2• 6 years ago

This really seems like whistling past the graveyard on google's part here. There's too much meat to the story for them to really do much more than obfuscate. The intersection of race and gender, ethical implications of big tech, the capitalistic pursuit of innovation at the expense of individual freedom. All of these look bad for google

theobeers• 6 years ago

And the examples of issues flagged in review that Jeff keeps highlighting—like Timnit’s alleged failure to mention recent work to reduce the environmental impact of large models—are themselves a bit worrisome. Jeff gives the impression that they demanded retraction (!) because they wanted Timnit and her coauthors to soften their critique. The more I read about this, the worse it looks.

omgwtfbyobbq4• 6 years ago

That was my impression as well.

Having said that, if Jeff were to make public the paper, criticisms of the paper, and improvements made to address the problems described in the paper, that could go a long way towards clearing the air.

acoard• 6 years ago

Yeah, put more simply, they pushed out someone in their Ethical AI department because they did not soften critiques against AI enough. They couch these in terms of rigour, but the substance of the problem has to do with her criticisms against AI.

Ultimately it makes the whole Ethical AI department look more like a rubber stamp for Google.

that_guy_iain7• 6 years ago

Jeff gives the impression that they demanded retraction (!) because they wanted Timnit and her coauthors to soften their critique. The more I read about this, the worse it looks.

I get the impression that she wrote a hit piece on Google and published with Google's name. For me, it's correct they demand a retraction. It's simply unprofessional to critque your company for something while not mentioning the work they're doing to combat that.

trhway4• 6 years ago

Google isnt publicly funded academic institution. Whatever they are doing, in particular publishing, is part of the business/PR. So if the management sees something not good for business it is a reasonable that they decided to not do it. If i were a shareholder i can see how i may have questioned why a person being paid $1M+/year (my understanding this is minimum what a manager in AI at Google would be making) for publicly disparaging Google.

Even more, it sounds like Google didn't ask originally for retraction, they just asked to take into account the newer research contradicting the paper - the thing that any researcher valuing integrity over agenda wouldn't refuse.

If somebody wants to do that research and publishing they just have to find another source of funding, i guess.

Anyway, the firing wasn't over the paper, the firing was over the unacceptably unprofessional reaction to it.

visarga9• 6 years ago

Jeff gives the impression that they demanded retraction (!)

In paper reviews you can often see reviewers asking the authors to rewrite, clarify, add extra experiments, add missing citations. It's all normal.

jariel7• 6 years ago

"Jeff gives the impression that they demanded retraction (!) because they wanted Timnit and her coauthors to soften their critique. The more I read about this, the worse it looks."

Representing a more truthful reality is not 'softening'.

It's only 'softening' for those who have an already accepted, extremist view, and for whom any evidence to the contrary doesn't help their arguments.

While initially sympathetic to the author - the more I read - the more I have completely the opposite view.

tylerl• 6 years ago

The communication doesn't give that impression; instead it says that the paper makes claims that ignore significant and credible challenges to those claims. Dean said that these factors would need to be addressed, not agreed with.

Publishing a transparently one-sided paper in Google's name would be a problem, not because of the side it picks, but because it suggests the researchers are too ideologically motivated to see the see the problem clearly.

Ironically, it indicates systemic bias on the part of the researchers who are explicitly trying to eliminate systemic bias. That's just a bit too relevant to ignore.

kwillets• 6 years ago

That confused me as well -- where I work we have a legal dept. approval for IP issues, and that's it. Academic review doesn't make sense in that context or time frame.

petulla• 6 years ago

This response really seems like gaslighting. He doesn't address her concerns and glosses over whether she was held to a different standard than others at GR.

bmitc2• 6 years ago

He was also extremely vague, perhaps intentionally, about what the issue actually was. His sentence about when the paper was submitted and approved and all that is impossible to parse and make sense of who did what and when.

ketzo• 6 years ago

Right -- he once again talks about "accepting" her resignation, when it reeeeeally just looks like they fired her. At the very least, she certainly feels like she was fired; why is that not mentioned at all? Even just, I don't know, "sorry we were abrupt?"

cbsmith5• 6 years ago

With such public communications, there really isn't much of a chance they'll be detailed and specific. It's why they are so rarely done.

pengstrom• 6 years ago

Increased scrutiny on minorites speaking up is a common historical occurance.

eachro2• 6 years ago

But in this scenario, shielding Google from liability is actually a primary concern given that Timnit is discussing ethics/bias. A paper on say novel transformer architecture, the lottery ticket hypothesis in a new setting, a new RL benchmark suite, etc is not going to expose Google to legal risk the way ethical AI research often can.

aabhay• 6 years ago

This. I have been arguing the unpopular opinion that most AI ethics work in corporate settings is not designed to empower real research. It was a matter of time before an actual researcher with an ethical compass was removed unceremoniously. Anyone in an AI ethics team at a large company — you need to know exactly what your job means to the company, because it isn’t safe.

oh_sigh4• 6 years ago

How would publishing a paper open a 3rd party up to legal risk? Research papers aren't laws, and it is chronologically impossible for a research paper to influence already on the book laws.

headsupftw• 6 years ago

Then what's the point of hiring her and people like her to work for Google in the first place? So that Google could claim that they have Ethical AI researchers and Google's AI research is indeed ethical?

lokar7• 6 years ago

In technical infra I saw papers rejected that were deemed to be not interesting enough (not a big enough novel contribution).

cavisne• 6 years ago

We know large language models are super important to google, and there are lots of competitors.

If they approved the paper the message would be "google thinks language models are a waste of resources and racist". There would be no academic debate on this topic as its been framed as woke and published by a militant activist, so any disagreement would be racist (see prior interactions between this researcher and other researchers [1]).

Thats why the standard process of publishing, peer reviewing, academic critique etc would not work.

Why would their researchers working on language models stay? when they can go to Facebook, OpenAI etc. Why would new researchers join?

[1] https://syncedreview.com/2020/06/30/yann-lecun-quits-twitter...

freeone30005• 6 years ago

Academic debate is, in fact, done through conferences and journals. You saying there can be no debate is a strawman position with no basis in reality. The idea that standard rigor cannot be applied to ethics research is absurd, and seems to insinuate that the entire field is absent discipline.

The proper response to her position would be to publish a response or critique. Attacking her entire field does nothing to further the conversation.

yodelshady2• 6 years ago

She's got solid STEM credentials. If google managed to hire a "woke militant activist" instead of what they wanted... is that really much better?

Demand the best from your multi-billion dollar corporations.

Aeolun3• 6 years ago

I have to assume both sides here are adults that can deal with criticism of their chosen discipline without immediately resigning, or not joining a specific company over it.

lallysingh2• 6 years ago

Also, when has feedback on a paper in this process been relayed via HR?

You only bring in HR protections to protect the company from a legal standpoint.

calf2• 6 years ago

The key sentence is the bullshit about a paper not offering mitigation as not helping. Since when is a scientific paper required to do that.

BeetleB5• 6 years ago

When I was in academia, it was not unusual for the referees to reject a paper for this reason. Of course, you are informed of that and always have the option to rewrite the paper to include that information.

In some cases, though, it's not simply a matter of listing it as other work in the Intro - you may need to incorporate it into your models, etc.

justicezyx• 6 years ago

If you are actually an academic researchers in an academia institution, and are exposed to a large scope of the dealings of the community, then you might find that professional academia is as political as corporations, if not more so...

dekhn4• 6 years ago

beyond reviewers or editors asking you to make changes like this (for example, "so-and-so just published <blah> which means that your sentence about <blah> is obsolete"), we're talking about research coming from a corporation. If one part of the company is trashing another part of the company, and it's based on stale results, asking to have the paper updated to include the latest results is reasonable.

contemporary3433• 6 years ago

Ditto in my past experience at Microsoft Research. Never an actual review of paper's merits, just IP and maintaining trade secrets if applicable.

dzdt• 6 years ago

Yes it is disingenuous for Dean to pretend that this was a normal process applied to a normal situation. Clearly whatever happened on that team, this latest round was not the beginning or even most important part. Gebru's letter mentions her threatening to sue Google previously, for instance. [1] The discussion about rigour in a conference paper or internal review is obviously a pretense.

[1] https://www.platformer.news/p/the-withering-email-that-got-a...

jariel7• 6 years ago

"Pubapproval hasn't been used to silence uncomfortable minority viewpoints until now."

This is sad gaslighting of a reasonable concern the team had.

Having to endure some external review for what could otherwise be sensitive material.

The inability for the SJW crowd to work reasonably within very reasonable terms, to then resort to aggressive tactics such as 'demand the names and opinions of everyone on the board' and then publicly misrepresenting the situation is going to lose you a lot of favour.

Every time I read one of these stories I immediately feel sympathetic to the individual, but then upon learning more, I feel duped and maligned for having been effectively lied to.

The doors are wide open for progress, those who take it to micro-totalitarian lengths are not doing anyone any favours.

mostafab3• 6 years ago

Black female scientist Timnit Gebru fired - the end of Google as a top AI research institution?

https://melwy.com/blog/black-female-scientist-timnit-gebru-f...

kazamaloo3• 6 years ago

Every paper we submitted went through a technical review as well as legal and IP reviews. They were along the lines of cite this, cite that, run these experiments etc.

What's different in her case is that you don't see the names of the people reviewing. Being the devil's advocate, she MIGHT have a pattern of aggressively attacking people who reviewed their work before. So they might have made the reviewers anonymous this time.

contemporary3433• 6 years ago

Similar experience to another current Google Brain researcher: https://twitter.com/le_roux_nicolas/status/13346019609729064...

Submitting conference papers last minute is... normal.

sjg0073• 6 years ago

This tweet is on point.

cyrus20• 6 years ago

At least when I was there, my papers were getting thoroughly reviewed and often had to make some adjustments before getting approval. Never occurred to me to make any demands from the reviewers or threaten to resign if my paper doesn't get immediately and unconditionally approved. Seems like she's asking for preferential treatment.

contemporary3433• 6 years ago

She's the one asking for an option to revise and discuss it. Management is the one demanding an unconditional retraction, with no recourse.

stjohnswarts2• 6 years ago

Why wouldn't you want to weed out bad papers as close to the source as possible to save company embarrassment and external people's time? If you see something wrong during a review why not push it back to the author before it does rounds outside the company? That would seem like a very bad practice to me.

creato• 6 years ago

I don't think the approval process is being used to enforce rigor in general, the (claimed) problem is the paper lacks rigor specifically in regards to claims about google's behavior.

Publishing a paper with a lack of rigor about some obscure mathematical technique isn't a problem for google (beyond some possible but unlikely mild reputation damage). Publishing a paper with a lack of rigor that says google is doing unethical things, when those things are questionably accurate, that is something google would (and should) have a problem with.

Whether the paper actually lacks rigor in a relevant way is not something I can comment on.

kaymanb• 6 years ago

Our aim is to rival peer-reviewed journals in terms of the rigor and thoughtfulness in how we review research before publication.

I can see how this might be frustrating for academics working within Google. The field already has systems in place for peer-review. While I admire the idea of Google holding their research to a certain standard, it also provides a mechanism for dismissing research that paints Google (the corporation) in a bad light. If a paper is good enough to pass an (external) review process, why should it not be published?

iab• 6 years ago

I think this is the crux, in that google is not academia; so naturally a band-gap would arise if your expectations were that it should behave like an academic institution. From their perspective, what requirement is there to publish work that is not well aligned with their goals (optics, financial, etc)?

htrp7• 6 years ago

The problem is also that Google's AI labs are treated more or less as academia, that's one of the big selling points for recruiting.

I'm in industry and am not allowed to even talk about the projects I work on, much less publish papers, it kills recruiting conversations with maybe 10% of the people at conferences.

danpalmer2• 6 years ago

If a paper is good enough to pass an (external) review process, why should it not be published?

There are many journals that have a poor track record on peer review, and in fact even those that have a generally good track record often have times that it famously failed.

Whether we agree or not, Google want to protect their reputation. Because not every journal will be up to their standards, ensuring that papers do meet a standard is going to be important for them.

They could do this by only allowing submission to certain journals, but this would require an understanding of each journal, its processes reputation, etc. Perhaps easier is just to review papers before they are submitted.

This does also allow them to protect potentially NDA'd info from being accidentally included in a paper given to external reviewers, which is (whether you agree or not) something they will clearly want to do.

stagger87• 6 years ago

If your company pays for your research, expect them to review it. This is not surprising and is par for the course in corporate research environments.

mrDmrTmrJ• 6 years ago

Great question! Here's a scenario, though I don't know if it's the case here:

Image that one set of NDA'd data was included in the paper, that painted Google in a bad light, but a different set of NDA'd data was excluded in paper, that painted Google in a good light.

I have no idea if that's what's going on here. But if some of the people who objected to publishing the paper thought this was happening, it may explain their objections to publishing the work.

bnj6• 6 years ago

While I’m a spectator to this unfolding story and am reacting to a pretty cursory overview of what happened, this has the distinct feeling that Google thought they were moving in for a checkmate by, in their words, “accepting a resignation”, only to have it — very predictably? - blow up in their faces completely.

Even if Google is acting presenting the chain of events faithfully, their final move to jump on an opportunity to remove the researcher and call it resignation seems so aggressive and incongruous that from the outside it makes it seem like this conflict is rooted in a larger and more difficult relationship that they calculated was no longer in their interest.

I’m wondering if the cost benefit analysis is still looking that way on the inside, because this move and the attention it’s causing is so contrary to their stated goals that I have to wonder if Google is committed to those goals at all. Others must be wondering exactly the same thing.

pm90• 6 years ago

This sums it up almost perfectly. The only thing I would add is that things like these tend to have a pattern. If a single person speaks up and you have multiple others claiming similar treatment, it significantly increases the likelihood of the workplace being hostile to such persons.

Also: Googles bar for hiring is extremely high. For multiple senior engineers/managers to corroborate this description is rather damning.

noitpmeder• 6 years ago

I don't see what else they should have done? This researcher made some ultimatum. As part of it, she said if Google refused, she would set a date to leave the company. Google obviously did not concede to her demand, and obviously didn't think it beneficial to let her be a thorn any longer, so took her up on her offer.

pseudalopex12• 6 years ago

She didn't offer to resign immediately. Fewer people would be criticizing Google if they just accepted her actual offer.

that_guy_iain7• 6 years ago

While I’m a spectator to this unfolding story and am reacting to a pretty cursory overview of what happened, this has the distinct feeling that Google thought they were moving in for a checkmate by, in their words, “accepting a resignation”, only to have it — very predictably? - blow up in their faces completely.

I wouldn't say this has blown up in their face. But to be honest, I've not seen that much drama about it than one post. Have I been missing something?

This seems a very solid move from Google. Someone tried to bully their way into getting information from Google they shouldn't give out. If you go to HR in confidence that confidence should be upheld. The fact someone had to go to HR for a peer review and the fact it was upheld states something.

If someone says "Do X or I'll quit" when you're not going to do X, it is a resigination and seems quite standard that they don't want such an active detractor within their company ranks.

I believe she basically sent an email to fellow employees basically telling them to stop working. That alone is a firable. That is completely nuts and unacceptable work place behaviour. The fact Google didn't fire her for that should speak volumes.

To me, from a HR point of view, this is completely the right move to make.

zaksoup3• 6 years ago

If somebody used HR as the mechanism to communicate feedback on the scientific rigor of research I was leading I would also find that concerning and objectionable. Human Resources is not peer review.

nebulous15• 6 years ago

If someone says "Do X or I'll quit" when you're not going to do X, it is a resigination

I mean, whether or not this stands up legally isn't worth discussing but I don't think this is typically how an ultimatum goes. Rather it's more along the lines of "this or I'll quit", "okay, we're not doing this", "right, I quit". It's not called calling your bluff for nothing.

I agree that her email to the Brain group makes her position at google almost untenable, but that's a separate question. However, it might make her choose not to fight them on the resignation.

sillysaurusx7• 6 years ago

This is little more than gossip unless you can point to something specific that people can judge for themselves. Saying “she basically did X” is a clever way to influence how people feel about her.

I’ll admit, I had negative thoughts till this thread. Now I’m not so sure. She’s definitely passionate, and one might say aggressive, but more and more people are saying that it’s extremely unusual for Google to demand a rejection for academic reasons rather than business concerns. Whether she was a good employee is kind of beside the point now.

bnj6• 6 years ago

I don’t think you’re incorrect on any particular point.

Just anecdotally, I don’t think a man sending a “do x or I’ll quit” email would be met with the same response. I think issues of race and gender are playing a role here, and that’s a shame, because while firing (accepting this resignation) of this researcher might make short term business sense, it seems like it does harm to Google’s long term credibility in trying to engage with reducing the barriers faced by (among others) women and people of color.

It’s a shame there wasn’t a pathway to keeping the dialogue going, for example by getting in touch with her and letting her know that some things she was asking for wouldn’t work, but that you could work with her to accomplish her goals in some other way.

From my privileged outside point of view, that seems like it could have been a more humane response which could have disarmed the conflict. I think opportunities to pursue options like that are de-emphasized when management are put in an all or nothing position.

A missed opportunity for leadership.

runako• 6 years ago

not seen that much drama about it than one post

This morning, there were front-page posts about this on the New York Times, Washington Post, Wired, Google, BBC, and Financial Times. It's being covered pretty widely.

karmasimida• 6 years ago

One thing that is missing here is the demands the said researcher had brought up to the leadership. Didn't see either party bring it up in their side of story.

quotemstr• 6 years ago

Did it blow up in Google's face? A bunch of perpetually angry people on Twitter are angry. So what? Google's actual operations are affected not only tiny bit by this episode. Outrage has no power.

Social media is not real life. Every company, Google included, needs to simply ignore social media outrage and focus on their mission instead.

Internet outrage only has power because companies give it power. Angry words and hit pieces from biased media do nothing on their own. If you don't let them make you afraid, you have no reason to be afraid of them. The internet screamed at Coinbase after the company showed their internal activists the door, but Coinbase is doing just fine. See? Nothing to be afraid of. Just get rid of troublemakers.

Aunche2• 6 years ago

I think that Google viewed her as a ticking time bomb, so they might as well detonate it early to try to contain explosion. She had threatened to sue Google in the past and was most likely gathering even more ammunition for her lawyers.

Edit: I'm not sure what this tweet is referencing, but it's an indicator of her attitude towards her superiors, I can see why they would want to let go of her.

https://twitter.com/timnitGebru/status/1331757629996109824

visarga9• 6 years ago

You might be on to something. The paper doesn't seem too controversial or especially amazing, the objections seem minor (some citations and updates), maybe it's just a pretext. Could be about a lawsuit.

drfuchs• 6 years ago

My researcher acquaintances at industry labs at IBM, Microsoft, HP, Xerox, ATT, Bell, DEC, Compaq, etc. never have had to have their papers reviewed internally before submitting them to conferences or journals. What's up with Google?

gcr3• 6 years ago

Until now, nobody on my previous team at Google had to have their papers reviewed internally for scientific rigour either.

(We have a publication approval process, but its purpose is to prevent IP leaks and maybe shield liability, not uphold scientific rigour standards)

lokar7• 6 years ago

I've seen Urs reject papers out of Tech Infra that he thought were not interesting enough

lallysingh2• 6 years ago

This is the AI Ethics team in a company betting their future on AI.

They know they have a bubbling ethics problem and they've screwed over their best chance to do anything other than completely fuck it up.

In history, this will be a wtf double facepalm moment.

kazamaloo3• 6 years ago

Technically you can submit it without pubapprove unless somebody rats you out and you might face some repercussions. Otherwise every paper is supposed to go through AI Ethics committee review along with other types of reviews.

bnj6• 6 years ago

I wonder how consistently these standards are applied, even zeroing in on Google. If it turns out that most people within the company don’t face a barrier from this process in the same way then that sounds like a really problematic symptom.

htrp7• 6 years ago

Apparently not very consistently, at least if you're Google Brain CA

https://twitter.com/le_roux_nicolas/status/13346019609729064...

dang2• 6 years ago

Previous threads on this ongoing story:

https://news.ycombinator.com/item?id=25292386

https://news.ycombinator.com/item?id=25285502

https://news.ycombinator.com/item?id=25289445

thisgoodlife• 6 years ago

one link says she resigned. The other one says she was fired. Which one is it then?

dang2• 6 years ago

It's fuzzy, and people are disagreeing about which aspect to emphasize.

exolymph• 6 years ago

People have argued both cases. The tl;dr is that she said "I will resign if XYZ" and Google was like "we accept your resignation effective immediately." Whether or not that counts as firing is a matter of opinion.

sp332• 6 years ago

The headline on the platformer.news article was updated to reflect the fact that she was fired.

ethanp61• 6 years ago

She made demands and gave a resignation date if those demands were not met. Google said that they could not meet those demands and accepted her resignation but because of her conduct (emailing hundreds of Googlers), the resignation date was made immediate.

fuzzylightbulb2• 6 years ago

Timnit: "if you don't give me x, then i am going to do y in a week."

Google: "we aren't going to give you x, so you can have y. in fact, you can have y starting right now."

i'd say she tendered a conditional resignation and they called her bluff and accepted it.

hartator2• 6 years ago

It sadden me that they make Jeff Dean work on these kind of issues. His map/reduce invention brought to the world a lot more than managing his coworker egos.

geodel2• 6 years ago

How do you know they are making him to this. It seems to me Jeff chose to accept leadership position at Google. So managing egos would be first class task to deal with for him.

tempest_5• 6 years ago

Exactly.

If managing people were the same skill set as computer science perhaps we would not be seeing this play out as it has.

This is seriously a massive failure on the part of Google to handle their own shit. I often wonder if these people are too old or out of touch to appreciate how this kind of thing may or may not blow up on social media.

To not consider how this might cause public relations issues especially when dealing with someone who has an established social media presence seems like a misstep.

On the flip side maybe they thought it was worth it. Weather the storm for a few days and twitter will move one. It's hard to say.

typon• 6 years ago

Jeff Dean did not invent map/reduce

foobarian• 6 years ago

OK then tensor flow

hartator2• 6 years ago

He did: http://cacs.usc.edu/education/cs653/Dean-MapReduce-CACM08.pd...

dekhn4• 6 years ago

Jeff Dean and Sanjay Ghemawat developed the system called MapReduce at Google and published a paper about it. They didn't invent the map or reduce functions, nor is the MapReduce system they published really about map or reduce functions.

The correct name is really MapShuffleReduce and the most important step is the shuffle, because it's a distributed sort. Of course, they didn't invent distributed sort, but combining these three concepts together in a distributed fashion and running it as a production system is really what was important.

brown9-22• 6 years ago

No one is forcing him to be the head of Google's Brain division or to manage other people.

wyldfire• 6 years ago

It appears that this was released intentionally by Dean. But in case it wasn't, or in case it gets altered/updated - it's been archived [1].

[1] https://archive.is/VpAN8

nova22033• 6 years ago

Timnit responded with an email requiring that a number of conditions be met in order for her to continue working at Google, including revealing the identities of every person who Megan and I had spoken to and consulted as part of the review of the paper and the exact feedback. Timnit wrote that if we didn’t meet these demands, she would leave Google and work on an end date. We accept and respect her decision to resign from Google.

This sheds some new light...

wilde• 6 years ago

Not really. People bluff in negotiations all the time. A normal response would have been, “I can’t do that. Are you still committed to leaving?” Instead Google went all scorched earth and it’s still unclear why. It also indicates that they don’t particularly value her in progress work in the lab since normally you want the 2 weeks or whatever to squeeze some wrap up work out of folks.

ghaff5• 6 years ago

People bluff but you generally have to be prepared to walk if your bluff is called. If I walk into a manager's office and demand 2x salary or I walk and the response is "we believe you are fairly compensated," you're now in a rather uncomfortable situation.

Which is of course not to say you can't have a more measured negotiation. But it can be hard to walk back from give me X or I do Y, especially if there isn't a lot of middle ground between giving you X or not giving you X.

vecter3• 6 years ago

It's fine to call bluffs. In fact, the definition of a bluff is something that, with perfect action on the other end, should be called.

Dig1t• 6 years ago

"Do X or I'll quit" is a childish way to negotiate, its akin to a child throwing a tantrum if they don't get their way. It's extremely unprofessional and not something a team player would do. I think its clear why they took her up on her offer.

neatze• 6 years ago

Some people don't have time for an bluff game, person choose the tone in negations with in power opponent without considering full extent of bluffing consequences.

ergocoder6• 6 years ago

This is an unfair take.

If an employee is adversary toward the company, the trade off needs to be made no matter how much valuable the employee's work is.

This is a non-issue tbh. She wants to leave. Company wants her to leave. They both agree to part way because the premises are fulfilled (i.e. company can't meet her requirements).

If you want to get technical, I'd bet her employee's status is still in tact for 2 weeks; she just doesn't have access to laptop and etc.

aylmao6• 6 years ago

I agree. Worth highlighting:

...if we didn’t meet these demands, she would leave Google and work on an end date.

Google's response to this was not accepting of her terms, it was to force an end-date for her —immediately— and lock her out of their systems ASAP.

jkbhzsdfkhjnzs2• 6 years ago

People are consistently leaving out the fact that she wrote an internal email encouraging her coworkers to reach out to congress about Google's behavior, at a time when big tech companies are being dragged in front of congressional hearings nearly every month

hobofan• 6 years ago

Google went all scorched earth and it’s still unclear why

Possibly because she made a personal problem into a team/department problem by asking their colleagues to stop working ("stop writing your documents because it doesn’t make a difference"). I couldn't imagine a company where such a call for work refusal wouldn't immediately lose you a ton of goodwill.

BeetleB5• 6 years ago

People bluff in negotiations all the time. A normal response would have been, “I can’t do that. Are you still committed to leaving?” Instead Google went all scorched earth and it’s still unclear why. It also indicates that they don’t particularly value her in progress work in the lab since normally you want the 2 weeks or whatever to squeeze some wrap up work out of folks.

I strongly encourage people to read some books on negotiations - as well as read up on legal ramifications to some negotiations.

Pretty much all books/courses on negotiations say: Ultimatums have their place, but are a minefield (i.e. they can blow up on you), and should be used as a last resort. From a negotiations standpoint, the response was adequate - which is why they all caution against using such an approach.

As for the 2 week thing, this is a convention, but not a requirement. In my company, it's not unusual for someone to be shown the door the same day they announce they plan to leave to another company (it's not the norm, but not at all unusual). The manager/company always ponders whether there are risks in keeping the employee for a few more weeks vs the gains, and this is the question Jeff pondered - that he did this is quite normal. Will the employee provide anything useful to us in those two weeks (e.g. handoff work to others, etc)? Could he/she cause problems (bad mouth people to fellow employees, steal IP, etc). If it's a disgruntled employee, they are usually shown the door the same day. In Timnit's case, it's unlikely there was any value in letting her stay for 2 more weeks.

I once intended to leave the company I was working for. The night before, I took out everything of (personal) value from my cubicle, as well as from my work machine. Only then did I have the discussion with my manager.

Having seen how she communicates and handles difficult situations, I think she really should read those kinds of books. Sometimes her behaviors are textbook examples of what not to do.

(Hint: If you're trying to influence someone, or a whole industry, you are negotiating, whether you choose to think of it that way or not).

bgribble• 6 years ago

According to her Twitter feed, Timnit was asking for the same transparency that normally goes with Google pub reviews -- where all the reviewers and their feedback are known to the reviewee. This red-stamp "review" did not follow the normal review process. Not a Googler, just going on what she says:

https://twitter.com/timnitGebru/status/1334881120920498180

mstg3• 6 years ago

Considering her previous attacks to anyone who disagrees with her and her previous public feuds, I'd say people may be afraid of being stamped as a racist/sexist because they had to reject her paper. She was a disruptive employee who threatened her employer and the employer didn't accept it. Why is Google obliged to keep her employed?

mikeryan• 6 years ago

You can read Timnit's letter/POV here https://www.platformer.news/p/the-withering-email-that-got-a...

visarga9• 6 years ago

According to her Twitter feed, Timnit was asking for the same transparency that normally goes with Google pub reviews

She has a Twitter army ready to set their aim on her reviewers and a track record, they can't reveal the names.

nx7487• 6 years ago

Yeah the way I see it, once someone makes an ultimatum, the relationship is destroyed, there's no trust. You can ignore the quality of her paper or the critiques from Google, they probably just decided to end the relationship because of this.

Imnimo• 6 years ago

This reads to me like Google felt that the paper painted some of their other technologies in a poor light, and wanted to insert language that made them look better. The way he describes their objections, they strike me as the sort of thing that is routinely addressed in the camera-ready version of papers by adding a few lines to the related work section. Not the sort of thing that a conference reviewer or an internal reviewer would reject a paper over.

Previously, we only had one side of this story. But if this is Dean's best spin on Google's side of the story, I'm very tempted to conclude they're in the wrong here. Obviously I don't have all the information, but the information I do have feels consistent with the idea that someone important at Google didn't like Gebru's paper for corporate-political (meaning making Google look good, as opposed to political-political) reasons, they tried to get Gebru to play ball, she refused, and now they have to back-project a justification in the name of "scientific integrity".

Unfortunately, I think this is a story where most people's opinions about who's in the right will be more informed by their previous opinions about Gebru and Dean than the narrower question of what happened with this particular paper. I'm probably even guilty of that to some extent myself, given that I'm a fan of some of Gebru's previous work.

asciident2• 6 years ago

I've seen these review comments before and I don't know if we can say from our position whether it's corrections that can be made for camera ready or not.

If a paper does not analyze the improvements from recent work, and just older work that has been surpassed (deemed inefficient by Dean), the new reanalysis might not be as favorable to the results as the paper proposes which means the paper is moot.

Or the second point is about bias in language models, but it sounds like these issues are mitigated in recent research, which means people are already aware and have solved a bulk of the issue being described in the paper.

But certainly it's possible that the paper's contributions stands strong even after accounting for the recent work that Dean mentions. In that case, it could be corrected for camera ready. But my point is that we can't tell right now without seeing the paper, and the relevant research that was omitted.

whimsicalism6• 6 years ago

But this review process had never been used for introducing comments of this sort to a paper ever before. Plus, there was no suggestion of change, just retraction.

skissane2• 6 years ago

they strike me as the sort of thing that is routinely addressed in the camera-ready version of papers by adding a few lines to the related work section

What I don't understand is why in the discussion nobody proposed amending the paper rather than withdrawing it. If Dean's issue was it didn't cite papers X,Y and Z, rather than demand a withdrawal, why didn't he just demand "I want you to amend the paper to add cites to X,Y,Z". And then, if Gebru and her coauthors were willing to add those cites, that would resolve it.

Indeed, from what I understand, "you should add a cite to X" is common peer review feedback, and a lot of papers get citations added due to requests from peer reviewers. So this isn't hugely different from that scenario.

It would seem that withdrawal over this issue would only make sense if Gebru and her coauthors refused to amend the paper to add the requested citations, but I haven't heard anything saying that she did refuse to do so. It isn't clear if the alternative solution of amending rather than withdrawing was ever brought up in the discussion by either party.

Not that I'm a researcher or anything, but if I was, and a superior told me "we need you to withdraw your paper because it doesn't cite X,Y,Z", my immediate response would be "How about I add the citations you are requesting and resubmit it with those additions?"

pseudalopex12• 6 years ago

Gebru complained about not being told what the criticisms were. That implies she was willing to consider feedback.

fatbird5• 6 years ago

In Timnit's account, she said she asked for specific feedback so it could be addressed and the paper published; this was denied to her and is part of her complaint over which she resigned.

quest88• 6 years ago

Maybe they did? We know, from Jeff's doc, feedback was given and then demands were made.

htrp7• 6 years ago

What I don't understand is why in the discussion nobody proposed amending the paper rather than withdrawing it. If Dean's issue was it didn't cite papers X,Y and Z, rather than demand a withdrawal, why didn't he just demand "I want you to amend the paper to add cites to X,Y,Z". And then, if Gebru and her coauthors were willing to add those cites, that would resolve it.

Unless corporate tells you to kill the paper and you need something that resembles legal cover.

xapata5• 6 years ago

Jeff's document says she submitted without asking for approval, so the request was to withdraw that unapproved submission. It is reasonable to interpret that as permission to revise and resubmit. She seems to have had her heart set on this particular conference and submission deadline.

jonahbenton• 6 years ago

Actually, the opposite. Speaking personally, I have been a fan boy of Dean's work on a number of fronts for a decade. I had awareness of her work but more superficially, and saw it as largely pissing into the wind. Reading hers more closely now, her papers and her communication, she has my complete respect. I better understand the way in which she was scaling this incredibly difficult mountain. Deeply, deeply impressive.

And Jeff has lost mine. Even if his comments are not just bad faith, his blindness borders on incompetence for the role he has. Yet another clever, blind white man.

I hope a position in the Biden administration can be found for her, and her vision can have a larger societal impact.

throawaydmg2• 6 years ago

Timnit responded with an email requiring that a number of conditions be met in order for her to continue working at Google, including revealing the identities of every person who Megan and I had spoken to and consulted as part of the review of the paper and the exact feedback

It always amazes how blatantly authoritarian these "woke" types are. I would not be surprised if the sole reason she wanted the identities of every consultant was to engage in some sort of witch-hunting and bigoteering[0]

[0] https://www.urbandictionary.com/define.php?term=Bigoteering

foota2• 6 years ago

I can imagine she felt helpless when her work was being blocked. Imagine you were going to start a project and was told someone in your org was against it and that you couldn't do it as a result.

kevinskii2• 6 years ago

That pretty much sums up every employee/employer relationship, and it isn't a bad thing.

throawaydmg2• 6 years ago

Maybe take a step back, consider the possibility that you may be wrong, and if you still think you are right, then attack their ideas and arguments, instead of acting entitled and demanding to know who said this or that.

Like PG wrote, the worst thing you can say about an idea is that it is wrong, you don't need to resort to personal attacks or x-isms.

Orthw2020• 6 years ago

That is the only reason.

If your entire career is built around solving a problem, then actually solving it is against your personal best interest.

bnj6• 6 years ago

Isn’t it pretty normal to ask to “face your accuser” and engage with feedback specifically, instead of an anonymous and unassailable red stamp?

dekhn4• 6 years ago

That's in court, or whatever, not academic or industrial anonymous peer review.

sidlls3• 6 years ago

In the context of peer review it’s common, or at least not uncommon, for reviewers to be anonymous. Personally I think it should be completely blind both directions to promote maximum objectivity.

Whether that applies at all in this case is a separate matter.

AnimalMuppet2• 6 years ago

It is. It's not a request you're ever going to get when you're dealing with the HR department, though.

Which puts you in a no-win situation when you suspect that HR is doing shady stuff against you...

Aunche2• 6 years ago

This tweet may explain why the reviewers do not want to be confronted:

Nothing like a bunch of privileged White men trying to squash research by marginalized communities for marginalized communities by ordering them to STOP with ZERO conversation. The amount of disrespect is incredible. Every time I think about it my blood starts boiling again.

https://twitter.com/timnitGebru/status/1331757629996109824

_diq5• 6 years ago

I would caution you by saying that no quote, paraphrase, or evidence of exactly what Timnit said.

Since this is Google's side of the story its in their interest to use words that would imply she's overbearing, or otherwise unaccommodating

(for all I know she could be all of those things, but this narration isn't sufficient evidence to prove that)

root_axis• 6 years ago

Woke politics is a red-herring. Google is typically cool with woke politics, as "anti-sjw" types are usually quick to point out. If you take it out of the culture-war context its a lot easier to see what this story is all about.

harrylove• 6 years ago

I wonder if stories like these should be off-limits for accounts that were created specifically to spread FUD. Several accounts just like this one appeared simultaneously in one of the other submissions as well. They added nothing to the discussion other than to make ad hominem attacks on Gebru. We’re seeing the same thing here.

edit: typo

brown9-22• 6 years ago

Do you work in an environment where design or code review feedback is anonymous? So why would it be the case for internal peer review of a paper?

qppo• 6 years ago

Yesterday the conversation was all over the tone of the email exchange, and my gut was regardless of the research it discussed that Google was probably alright to choose to accept that researcher's resignation.

Now I think I was wrong, Google looks like they're full of crap. If the research doesn't pass muster I'd like to read it and pass my own judgement. I'm guessing the tone was justified.

javajosh• 6 years ago

What's even worse is that Google ignored Timnit's attempts to figure out what went wrong, handing down a decision by fiat. I am surprised that this doesn't violate Google's standards! It would certainly violate mine.

OTOH Timnit sounds like a down-the-middle SJW type, which means that every conflict is about identity, privilege, and so on, which honestly hurts her message. To me, its enough that Google shut her paper down and wouldn't tell her why or give her even a chance for a different outcome -- perhaps it is, ironically, her SJW nature that makes it risky for Google to engage her in the way she wants to be engaged, because a deeper conflict cannot look good for Google in the current cultural climate. (E.g. sounds like she's got strong intellectual integrity points, and weak SJW points to make about Google, but they couldn't engage the former without also dealing with the latter.)

visarga9• 6 years ago

ignored Timnit's attempts to figure out what went wrong, handing down a decision by fiat

Funny, because she mobbed Yann LeCun and after she obtained a forced apology from him and an invitation to a civilised and serious discussion she said "this is not worth my time".

Maybe it wasn't worth their time.

google234123• 6 years ago

If oyu are paid by Google, Google has a right to vet your external work related to your position.

zemo• 6 years ago

ok google234123

choppaface• 6 years ago

It's frankly unprofessional for Jeff Dean to post only his email in this document without providing more context. The news media has in cases provided balanced coverage that included the email from Timnit that prompted Google's action: https://www.platformer.news/p/the-withering-email-that-got-a...

This post from Jeff Dean simply underscores that he has failed to balance the need for diplomacy with the research thesis of his own research group. I'm not saying he's being malicious, but incredibly tone deaf. While I appreciate he gets "attacked" at nearly every talk (at one retinopathy talk I saw him grilled for 10 minutes on race), he's going to continue to get this sort of attention until he can stop being the Googler who wants to tell you why their view is right.

saagarjha5• 6 years ago

I mean, this is obviously a place where Jeff Dean is providing the voice of Google on the topic. Why would they provide balanced context?

hackcasual2• 6 years ago

A large part of the criticism directed towards google is that they weren't particularly transparent in their handling of this situation. This is making that look worse.

gundmc• 6 years ago

It seems reasonable to choose to publish something you wrote yourself (and presumably cleared with legal/PR) and not publish something written by someone else with an intended internal audience. It would be WAY worse if they unilaterally published Timnit's private emails publicly.

bagacrap2• 6 years ago

Indeed, it's usually considered poor form to forward emails to broader mailing lists.

SpicyLemonZest3• 6 years ago

I'm not sure what you mean by "unprofessional" here. In my eyes, a vague statement that they had some irreconcilable differences but she's still a great researcher is nearly the archetype of professionalism. It would be extremely unprofessional for him to publicly criticize an email Timnit sent privately to the Women in Google Brain group.

nullc2• 6 years ago

Excerpts were already being leaked on HN.

The 'email from Timnit' doesn't exactly look good... I would expect a manager posting company lists in that manner to be dismissed (or a junior employee to receive some mentioning about professional conduct, if not dismissed).

I don't believe anyone has published the ultimatum email, which I think would be more accurately described as what precipitated the outcome.

aylmao6• 6 years ago

I hope Timnit finds it [1]. Google has it in their email servers but I doubt they'll publish it.

[1] https://twitter.com/timnitGebru/status/1334881120920498180

Footkerchief3• 6 years ago

The followup does not answer the major questions raised by this part of Dean's original email:

Unfortunately, this particular paper was only shared with a day’s notice before its deadline — we require two weeks for this sort of review — and then instead of awaiting reviewer feedback, it was approved for submission and submitted. A cross functional team then reviewed the paper as part of our regular process and the authors were informed that it didn’t meet our bar for publication and were given feedback about why. [...] We acknowledge that the authors were extremely disappointed with the decision that Megan and I ultimately made, especially as they’d already submitted the paper.

When it was "approved for submission," was that approval final and actionable, or some kind of conditional approval? Is demanding a retraction after an approval the normal way it works at Google, or was this an unusual occurrence that Gebru was right to question?

chrisdinn• 6 years ago

Why doesn’t Jeff just suck up his pride, apologize and hire her back rather than write increasingly thin rationalizations for his reaction?

High level research (and engineering) will involve egos and you should expect this kind of push back when you stop someone from publishing. Nothing here justifies how he handled it

Of course, I think many of us have seen this reaction before, maybe even done it ourselves. It’s bad for everyone, don’t do it Jeff!

schoolornot• 6 years ago

Ok, put another way. What interest does Google have in re-hiring her? She left on a bad note and brought tremendous negative PR. The company isn't losing business over this as the treatment didn't rise to abuse or harassment. Her team will operate just fine - everyone is replaceable.

Apparently Google has varying degrees of transparency when it comes to decision making. To be expected at a large company. There are some things my company & boss do that seem abnormal and out of character. You have to go with the flow sometimes and shut up. It seems like this person had some kind of utopian mindset where she felt management was obligated to explain everything around the blocking of her paper.

I doubt a guy like Jeff who wants to do tech all day has a vendetta against black women that want to publish papers.

What a way to go.

rajangdavis• 6 years ago

Had Gebru not raised an ultimatum, what would have been the consequence of the paper not passing internal review be? It sounds like she submitted the paper anyway, would there have been consequences to those actions?

visarga9• 6 years ago

asfadfdsa• 6 years ago

There are a lot of people commenting that she didn't actually resign. I agree, but it sounds like the conversation went like this:

employee: I'm not happy about x, y and z. If you don't do those, I'm going to quit.

manager: well we are not going to do those, so thank you for your time. We accept your resignation and would like it to start immediately (i.e. you're fired).

If you are gonna tell your manager that you plan to resign if a condition isn't met, then what do you expect them to say if they don't plan to fulfill that condition? It sounds like she was expecting them to say "Hey, well we don't want to meet your demands, but sure, we're happy to have a disgruntled employee around here, so feel free to stick around, or you could just quit on your own timeline, no sweat".

I suspect that many people would be fired on the spot for threatening to resign, so don't threaten it if you aren't okay with that consequence.

reikonomusha2• 6 years ago

What if the language weren’t so direct? For example,

If I can’t know who is providing reviews to my work, it’s difficult for me to imagine continuing to work as a part of this group.

Many people would still describe this (erroneously) as an ultimatum even though it could be bleakly summarized in the same vein as a true ultimatum.

SpicyLemonZest3• 6 years ago

We don't know the exact language, but by Timnit's account she made a numbered list of demands, which is pretty inherently direct.

fdasfsdafsa2• 6 years ago

It's more nicely worded to be sure, and definitely less aggressive, but it still says the same thing, which is "I'm gonna be an unhappy employee if I can't find out who reviewed my work". If they aren't willing to tell her who reviewed her work (which they may or may not have valid reasons for doing, but clearly they don't want to do), then they are dealing with someone who is going to be an unhappy employee since their conditions won't be met. Sure, not a resignation, but if your employer doesn't think you'll be a happy employee, why keep you around?

In any case, she has been very vocal on twitter and has not seemed to deny that she gave some sort of ultimatum. If she didn't give an ultimatum, it would only make Google look worse, so why not mention that on Twitter (given that she has tweeted probably 100 things about this incident in the last two days)? Given the absence of a denial, I'm going to assume that it was worded as an ultimatum.

ghaff5• 6 years ago

Yes. I probably didn't use the word "resign" as such but there was a time in my career when I went to my manager's manager or maybe another level up the chain and basically said I could not work for my direct manager. I got my way even though it involved working in a bit of a backwater for a time. But I was prepared to leave if I didn't get my way. (Didn't hurt that I knew said manager was a bit on the outs with the exec who I knew quite well.)

nebulous15• 6 years ago

manager: well we are not going to do those, so thank you for your time. We accept your resignation and would like it to start immediately (i.e. you're fired).

The thing is, there's a big difference between resigning and being fired for cause, even if both end with you not working at the company anymore.

fdasfsdafsa2• 6 years ago

IANAL, but my best guess is that she was just let go without cause. Your employer can fire you at any time and technically doesn't need a reason. Typically, "terminated with cause" is a specific thing where they fire you and give a specific reason (e.g. stealing) that might have bearing on whether you receive unemployment benefits, accrued vacation, etc. It's hard to imagine that they fired her in that way and that she was just plain-old-fired (there's a reason for it, but not legal cause).

UncleMeat9• 6 years ago

There are three different discussions happening all at the same time, which is muddying things.

1. Was the treatment of her research in internal review reasonable?

2. Was terminating her employment reasonable after she sent the (now public) email to the women-and-allies brain listserv?

3. Was the end of her employment at Google a resignation or a firing?

To me, (3) is by a wide margin the least interesting part of the story and all of the discussion here is missing the point entirely. Whether she was fired or resigned has zero bearing on whether (1) and (2) show reasonable or unreasonable actions.

visarga9• 6 years ago

For what it's worth, I have read the abstract of the paper and discussions around it and I have seen more than one person rate it as not very interesting. Why is all this fuss about a paper restating common knowledge? For example, they say datasets need to be filtered for bias, and that large models consume ... duh ... a lot. We already know that, where's the new shiny architecture for fair modelling?

link to abstract and discussion - https://old.reddit.com/r/MachineLearning/comments/k69eq0/n_t...

driverdan• 6 years ago

As a manager I would only do this if I really wanted to fire the person already. For employees I care about I would give them an out.

Part of being a good manager is understanding your employees and helping them succeed. If someone makes a statement like this in the heat of frustration it doesn't necessarily mean they will actually quit. If they're a valuable member of the team you should present them with an opportunity to save face and remain.

To me this seems like taking an opportunity to fire someone they already wanted to get rid of. Either that or a bad manager who wanted to flex their power as a threat to the rest of the team.

spdionis• 6 years ago

Given the employee's general attitude, I find it easy to believe her manager was already not happy with her.

See her previous (potential) legal troubles with Google that she even acknowledges herself.

I was in the middle of a potential lawsuit for which Kat Herller and I hired feminist lawyers who threatened to sue Google (which is when they backed off--before that Google lawyers were prepared to throw us under the bus and our leaders were following as instructed) and the next day I get some random “impact award.”

https://www.platformer.news/p/the-withering-email-that-got-a...

PragmaticPulp• 6 years ago

As a manager I would only do this if I really wanted to fire the person already. For employees I care about I would give them an out.

One of the more difficult lessons I learned as a manager: Once a team member starts giving ultimatums in order to get their way or override decisions, it's not in your best interests to give in for the sake of keeping that employee. The obvious exception is if you realize you were actually wrong from the start, but reversing decisions for the sake of caving to someone's demands is a problem.

If someone is so ready to quit that they'll flaunt it to the company, it's doubtful that reversing a single decision is going to suddenly make them happy again with their employer. Worse yet, it sends a message that threatening to quit is the way to get what you want from the company. Once you validate the strategy, you will get a lot more of it.

Unfortunately, if someone already has one foot out the door and has been complaining openly (even on Twitter, in this case) about their employer, it's best for everyone to go their separate ways. From there, focus on identifying and fixing any underlying problems to minimize the chance of this happening again in the future.

guantanamo_bob• 6 years ago

This debate over whether she was fired or resigned is distracting from the real issue. Why isn’t Google willing to listen and work with Timnit? What are the issues she’s raising? These are more important questions, fired/resigned is just an easy thing to be outraged or give leeway over.

friedman23• 6 years ago

Why isn’t Google willing to listen and work with Timnit?

Maybe she isn't that important for the company? Why bother working with someone making demands if they aren't worth your time. Google might have actually thought she was detrimental to the company and wanted an excuse to fire her and she gave it.

throwaway19592• 6 years ago

I suspect that they were looking for a way to separate themselves from her. I was very upset when I read her attack on Yann LeCun. At least in my eyes, she was hurting Google brand. I read her tweets and she seems to see racism in every turn. In my opinion, that was a behavior of a political activist, not an AI researcher.

coffeemug• 6 years ago

TL;DR: Jeff and Co got sick of Timnit's woke bullshit, pushed back, she threw around some ultimatums, they called her bluff and pushed her out. Personally I'm glad that Google is finally cleaning house. More of this plz.

courtf2• 6 years ago

Wait, I thought the tech industry was literally North Korea, and as long as you sing the woke anthem every morning, you could never get fired?

courtf2• 6 years ago

Google's got five more HR reps working on the case. They got them working in shifts!

geodel2• 6 years ago

I think Google can't be force to admit that when postmodernist racial theory ideologues are in the research / leadership ranks this kind of review is necessary. So Google is just going to say this is how we do it (or did it always).

oivey2• 6 years ago

I would be absolutely furious if a manager blocked my work from being published. Even a manager who has worked as a researcher, in my experience, significantly lacks the expertise to be making such judgments. A research manager’s job is to be familiar with work across an entire portfolio, so they will be necessarily less knowledgeable than individual senior researchers. Presumably Google generally also feels this way seeing as this appears to be the first case where prepublication approval was denied for content. I would never work somewhere where management had such a lack of respect for my own judgment as a researcher.

visarga9• 6 years ago

Then you probably have to get hired at a corporation that never puts its own well being above the rest.

oivey2• 6 years ago

No, in my experience, you are wrong. Becoming an industrial researcher trades some independence for money, steady project funding, and the opportunity to more directly impact the world. In return your employer gets first or sole access to your research in order to make their products better and more profitable. Clueless suits forcing you to degrade the integrity of your work by demanding you soften your message or by puffing it up with PR need not be part of the equation. This case is over a research paper, not a press release. Smart companies know that employing researchers that are honest people rather than sycophants is good for their bottom line.

Probably, even Google knows this, and this is just a mistake. If I was in the position of this researcher, I’d threaten to quit, too. Google’s actions suggest it wants to appear to care about ethics in AI rather than actually caring. That’s not a environment conducive to her to have any impact, so why stay? Clearly she’s not going to have a problem finding a job elsewhere.

supergirl• 6 years ago

sounds to me that google hired this ethics researcher but now they don’t like what she found

jl2718• 6 years ago

If she submitted, does that mean we are going to be able to read it? Is it on preprint?

spinningslate• 6 years ago

I don't know Jeff Dean. I have read some of his work, watched some of his presentations. He seems a credible bloke.

This, though, looks and feels like thinly-veiled retroactive and pretty unconvincing PR. It's short on detail and appears somewhat at odds with several points from Timnit Gebru's resignation note [0]:

- Dean says the paper was reviewed by a "cross-functional team". Gebru says she received the feedback through a "privileged and confidential document to HR"

- Dean says the paper was submitted for review on the day it was due to be published; Gebru says they had notified "PR & Policy 2 months before".

- Dean suggests the feedback was due to the paper not highlighting mitigating work for some of the limitations the paper was exposing. That seems like a very normal part of the research process. Why, then, does Gebru claim that she was told that a "manager can read you a privileged and confidential document" and that no other recourse or exploration of the feedback was permissible?

The only thing we know from the outside is that reality will be far more nuanced and complicated than the tidbits that leak out. Even allowing for that though - and reading some of the related comments here - Google isn't coming out of this well at the moment.

[0] https://www.platformer.news/p/the-withering-email-that-got-a...

EDIT: Fixed spelling of Timnit Gebru's name.

sillysaurusx7• 6 years ago

Dean says the paper was submitted for review on the day it was due to be published; Gebru says they had notified "PR & Policy 2 months before".

Wait, really? That’s an important detail if true. The one-day timeline was a central part of the narrative surrounding this story. Notifying them two months ahead of time makes this a completely different situation.

I’m a bit skeptical. Could this claim be verified somehow? Since it’s very public news at this point, we may as well try to be rigorous.

usmannk• 6 years ago

From what I read it sounds like Timnit cleared the general idea of the paper 2 months before. The paper itself seems to have been submitted for approval 1 day prior to the deadline. Another HN commenter says that submitting papers for approval "hours" before the deadline is common at Google.

mdanger007• 6 years ago

Your honor, I didn't beat my wife. But if I did, she deserved it.

tfehring3• 6 years ago

Dean says the paper was reviewed by a "cross-functional team". Gebru says she received the feedback through a "privileged and confidential document to HR"

I don't think these two takes are at odds with each other at all. It sounds like a cross-functional team reviewed the paper and produced that "privileged and confidential document" but Gebru didn't find that document sufficiently detailed.

kevinskii2• 6 years ago

Gebru says she received the feedback through a "privileged and confidential document to HR"

I agree that Google isn't looking good at the moment. But if I had a colleague who seemed intent on finding ways to place the company in legal jeopardy, then I too would avoid direct communication whenever possible.

aphyr• 6 years ago

Dean writes:

Our aim is to rival peer-reviewed journals in terms of the rigor and thoughtfulness in how we review research before publication.

But Gebru writes that HR and her management chain delivered her feedback in a surprise meeting where she was not allowed to read the actual feedback, understand the process which generated it, or engage in a dialogue about it:

Have you ever heard of someone getting “feedback” on a paper through a privileged and confidential document to HR?

A week before you go out on vacation, you see a meeting pop up at 4:30pm PST on your calendar (this popped up at around 2pm). No one would tell you what the meeting was about in advance. Then in that meeting your manager’s manager tells you “it has been decided” that you need to retract this paper by next week...

And you are told after a while, that your manager can read you a privileged and confidential document and you’re not supposed to even know who contributed to this document, who wrote this feedback, what process was followed or anything. You write a detailed document discussing whatever pieces of feedback you can find, asking for questions and clarifications, and it is completely ignored.

(from https://www.platformer.news/p/the-withering-email-that-got-a...)

I've been through the peer review process at Physical Review Letters, SIGMOD, and VLDB. You get a document containing all the reviewer's comments, plus a metareviewer's take on the overall decision and what has to change. You can engage in dialogue with the metareviewer, including a detailed response letter justifying your choices, highlighting things the reviewers may have missed, and explaining where you plan to make changes. You get additional rounds of comments from the reviewers in light of that letter on later drafts.

I'm not a Googler, and I have no idea what the standard review process looks like there, but what Gebru describes does not sound at all like peer review. I also note that Dean does not contradict Gebru's account of the meeting or feedback process. If I had a paper rejected in this fashion, I would also demand to know what the hell was going on and who was responsible.

This feels off.

xiphias23• 6 years ago

I think the person who submitted feedback in privileged and confidential way made a great choice. I expect more people taking this route in the future actually, people are scared in the current political athmosphere. Look at what Jeff Dean is getting, even though he didn’t do anything bad.

A manager in ethics shouldn’t ask Google to break the law by not providing confidientality that was requested.

visarga9• 6 years ago

He took one for the team, kept them safe.

asciident2• 6 years ago

I would certainly be fearful of providing an honest review of a paper championed by such a powerful figure at Google who could get me fired and unemployable with a tweet. Revealing the reviewers and throwing them under the bus would have been the end of any honest reviewing at Google. Dean made the right move here.

joshuamorton9• 6 years ago

A manager in ethics shouldn’t ask Google to break the law by not providing confidientality that was requested.

It is not implied anywhere that the reviewers requested confidentiality. It is certainly not implied that Google would be violating the law to rescind that confidentiality (and, to be clear: they almost certainly would not be).

dodobirdlord6• 6 years ago

It does feel off, and it’s easy to read as Google trying to suppress her findings. But there’s a simple charitable reading as well. Gebru has recently and very publicly developed a reputation for behaving with hostility toward colleagues. Google is a company that prizes (or at least claims to prize) psychological safety of employees. I can see all of this being the chain reaction caused by a number of her coworkers expressing that they were unwilling to give an honest opinion about her work if she would be able to tie it back to them. If an ordinary employee caused their coworkers so much concern they would probably already have been dismissed, but Gebru is especially talented and was high-ranking.

I imagine some reviewers extracted agreement from management before giving negative feedback on this paper that the “anonymous” in “anonymous feedback” was a promise. This explains the unusualness of the situation, why the feedback flowed though a special HR channel, why specifically management was unwilling to let her have a written copy of the feedback (that could be closely analyzed to de-blind the reviewers), and why management accepted her resignation and the resulting fallout rather than agree to de-blind the reviewers.

deeviant• 6 years ago

I feel that Jeff Dean is a much better engineer than he is a leader.

temp8964• 6 years ago

This whole sh*t show only proves certain kind of people are extremely toxic.

johncena332• 6 years ago

I've read her email. In my first job, right after college, I have also sent an inflammatory email (nothing compared to the email she has sent) to my manager (not a whole group of coworkers). And I got seriously reprimanded for it by my then manager. Even after a decade I cringe when I remember the email I wrote. I have no idea why people think it's okay to send emails like this and expect not to get fired.

apta5• 6 years ago

It's because of a culture that praises playing victim, no matter how successful you are, and regardless of whether or not they're telling the truth.

greatwhitenorth• 6 years ago

Every "woke" company in SV is going to go through this. I really want to know how "diversity' will help someone build a software company say a CRM app. How does a skin color/race help with that? If ideas of a different race matter, why should they be employees and not found through user testing/customer feedback? Coinbase, Google, keep it rolling.. No fully remote company has to go through this bullshit.

paijiut• 6 years ago

I don't know why Google is bothering to say anything. It's pretty obvious that those who support the person who left have anchored their opinions and won't change under any circumstances, and those defending Google are anchored as well.

themacguffinman• 6 years ago

While that is likely true, there's value to staking out your position clearly. It reduces speculation and can help you at least claim a side in a fraught conflict that could quickly turn both sides against you.

tinyhouse2• 6 years ago

In her email she claims that she is constantly being dehumanized. This is unacceptable to someone in her amazing position and to be honest sounds like she is a narcissist. I think Google couldn't wait to get rid of her (cannot blame them, she wanted to sue the company a year prior, also "represented" the company really bad on social media) and her email with demands was their opportunity. I don't think she would survive in any company. She is better off starting her own company or going to academia. I don't feel bad for her, I'm sure she already got multiple offers. Not from FB that's for sure though :)

apta5• 6 years ago

This is the problem with hiring SJWs. They will claim being victimized, racism, etc. the moment they don't have their way. I'm sure many people would have loved to be in her position and have her salary and benefits. Some people can't have enough it seems. "Wokeness" is a disease, the goal is to keep getting more and more power behind the guise of "equality".

Hopefully more employers will reject hiring "woke" people after seeing the trouble they're causing.

rebelos• 6 years ago

I suspect you're being downvoted because this post sounds like a denial of very real issues of sexism and racism.

However, the suggestion that employers will work to identify and preemptively reject employees with an SJW bent is not only sound but almost certainly already underway.

freediver• 6 years ago

Funny that Jeff Dean would write and share this in a google doc. After 20 years of being a driving force for the web, there was no other easier, more appropriate tool avaialable to a senior google employee to share his opinion online.

angry_octet5• 6 years ago

After reading Jeff Dean's response, I can only come to the conclusion that Timnit Gebru acted like a primadonna. It is completely normal to have to obtain prepublication signoff on material before it is submitted to a conference (and manager signoff even before abstract submission). Given the breadth of experience at Google, it seems strange not to avail yourself of this. Demanding to know the identity of reviewers is absurd (no journal would tolerate that) and deeply unprofessional. Making it an explicit ultimatum was her decision. Denigrating the entire area of research at Google on a large mailing list is the action of someone who wants to be terminated.

People claiming that the deficiencies in the paper are minor and wouldn't be blockers obviously have little experience submitting to academic journals. Other parts of Google doing deeply technical work probably don't have the same level of review as the Ethical AI group -- for obvious reasons.

There is usually a long back and forth -- there are even memes about the infuriating comments from "Reviewer 2" [1][2]. Omitting to mention argument-obsoleting developments in the field (from your own lab!) is more than enough to send you back to extensive redrafting.

To be clear on terminology -- a retraction is an academic black mark, and occurs to a paper after publication, usually for reasons of research misconduct. This is not an instance of that.

[1] https://twitter.com/redpenblackpen/status/113344056990719590... [2] http://jasonya.com/wp/wp-content/uploads/2016/01/PowerRespon...

joshuamorton9• 6 years ago

It is completely normal to have to obtain prepublication signoff on material before it is submitted to a conference (and manager signoff even before abstract submission)

To be clear, both of these had been received for the paper in question.

angry_octet5• 6 years ago

That isn't the impression I had from reading the principal's statements, to wit:

Unfortunately, this particular paper was only shared with a day’s notice before its deadline — we require two weeks for this sort of review — and then instead of awaiting reviewer feedback, it was approved for submission and submitted.

It seems like someone short-circuited the review process and submitted without review to meet a deadline, with a post-submission review. When this occurred, it was not all green lights. The expectation is that you then pull the submission and, post review, submit elsewhere. After all, conferences are all virtual now, it is not a hardship to submit elsewhere.

If you have other information as to the facts or the internal process do tell.

mattdeboard2• 6 years ago

I think, given the top comment on this post, your comment is far too credulous about Google's incentives being aligned with scientific rigor alone.

angry_octet5• 6 years ago

To be sure, Google has other objectives. But the people in this area are essentially academics, and I'd expect them to have a similar process given their backgrounds. The papers from Google are routinely of high calibre.

The non-research motive is arguably to control the narrative about using ML (I admit I still choke at calling it AI) and big data techniques. I fail to see how this is advanced by having a very public spat.

I've no doubt some senior exec decided she had to go, but I don't think it is because of any intersectional reason, or to cover up any particular publication, but that she wasn't seen as an asset to the company. Strategies for negotiating exercise of power are very different at Google to social sciences academia (or twitter, which is engorged with righteousness over this), which she seems not to have grasped. There are few enfants terribles in corporate tech that don't have controlling stock.

bmitc2• 6 years ago

Gebru's email to team members: https://www.platformer.news/p/the-withering-email-that-got-a...

Tweet explaining how she "resigned": https://twitter.com/timnitGebru/status/1334343577044979712?s...

An external reviewer of the paper: https://www.reddit.com/r/MachineLearning/comments/k69eq0/n_t...

I think you should be more open about what potentially actually happened because your first sentence seems very accusatory without proper information, the very bias that Gebru speaks of. According to her email that Dean references in his, her "feedback" for the paper was given in the form of a confidential HR document. That seems highly nonstandard for a review process supposedly only concerned with scientific rigor and is not addressed at all by Dean's statement. Further, his statement clearly, likely intentionally, muddies the water about what the actual sequence of events were that led to the paper being created, internal reviewers originally being notified, when the submission happened, who approved it, who submitted it externally, who asked for this to be retracted, and when and in what order all this happened. The fact that his statement, that was apparently meant to clarify these exact proceedings, makes it even more vague about what actually happened seems pretty damning to Dean's case, in my opinion.

Both Gebru's and Dean's statements have very little overlap in flavor and in facts, so it's pretty apparent something is going on here that is nontrivial and abnormal.

1234fdsazxcv• 6 years ago

I would agree with this assessment. It seems people think this is like academia -- it's not. There are no guarantees of tenure at a private company, even one with a research culture. If she wants that, then give up the paycheck and go back to academia. Google doesn't owe her anything. (I bet she'll get a decent settlement.)

AI ethics are important and I'm willing to bet that the Google execs want to know about biases and other problems in their products and probably don't care that much what is published. They probably don't want high maintenance researchers that cause them headaches and are obsessed with microaggressions, etc. I expect that this stuff has been a distraction for too long and they are happy to be rid of this and happy for the signal it sends across the company.

I have little sympathy or concern for Ivy educated elite in-fighting. There is probably a huge number of qualified people just dying to have the opportunity to work there, have access to those resources and do research.

gowld• 6 years ago

Whichever side's account you read, that side looks correct. This is not surprising.

There's a classic documentary about this phenomenon: https://www.youtube.com/watch?v=9UiF7BsC1Ig

angry_octet5• 6 years ago

That's a very snide comment. I've read her tweets and other Brain tweets (eg, from mmitchell_ai, though she deleted them, which seems wise as they were angry). I don't take any of them at their surface as being truthful. I think we're only seeing the 10% of the iceberg. But I don't find Gebru convincing.

jkbhzsdfkhjnzs2• 6 years ago

"So if you would like to change things, I suggest focusing on leadership accountability and thinking through what types of pressures can also be applied from the outside. For instance, I believe that the Congressional Black Caucus is the entity that started forcing tech companies to report their diversity numbers. Writing more documents and saying things over and over again will tire you out but no one will listen."

People are forgetting about the part where she basically encourages her colleagues to talk to congress, at a time when tech CEOs are regularly being hauled in front of congressional committees. At the point when that is written, this clearly is an adversarial relationship between her and Google. And it wasn't Google that made it adversarial

I couldn't imagine writing something like that and keeping my job

rirarobo• 6 years ago

I believe this excerpt is actually what led to her firing on such short notice. Actually, I find it surprising that this is going largely unnoticed.

Jeff's emails, which were surely also crafted with sign off from PR and legal, attempt to frame this "resignation" as an issue of the paper's content, as well as the inability to meet Timnit's demands. However, this framing does not seem to align with the experiences of other Google researchers, and does not explain why she was terminated in this particular manner. If that were really the case, why didn't they just wait for Timnit to return from vacation, try to resolve the issue, and if unsuccessful, accept her resignation? Why not try to deescalate and handle the situation more tactfully?

Timnit's account of the email she received also seems to confirm this:

"we believe the end of your employment should happen faster than your email reflects because certain aspects of the email you sent last night to non-management employees in the brain group reflect behavior that is inconsistent with the expectations of a Google manager. As a result, we are accepting your resignation immediately ..." [1]

So, what aspects of that email triggered such a prompt termination? Well, Jeff's first email says that Timnit was telling employees to "stop work on critical DEI programs", and I've similarly seen some comments focus on the part of her email that reads, "What I want to say is stop writing your documents because it doesn't make a difference," but Timnit is clearly not saying to stop DEI work, she's saying to "focus on leadership accountability" and to apply external pressure.

That seems to me to be the real issue, and the sort of thing that would trigger pressure from legal. The fact that the tone of the email is frustrated, and it airs some dirty laundry, would not necessarily result in firing, if leadership really wanted to keep her on, they could work something out. According to Timnit, her own manager was not informed, she's well liked by her academic community and her Google reports (who are expressing their dismay on Twitter), and Jeff has regularly praised Timnit in the past. So, the mention of congressional pressure in her email really seems to me to be the real catalyst here.

[1] https://www.twitter.com/timnitGebru/status/13343647354463313...

bagacrap2• 6 years ago

That paragraph, in hindsight, sounds like foreshadowing. This news cycle is her own "pressure applied from the outside" because she got tired of writing papers.

Seems pretty transparent.

deeeeplearning• 6 years ago

A lot of people mistakenly think Google is part of their family and has incentives other than making profit and avoiding bad PR. It's not really a surprise they took the first opportunity they could to fire someone who has in the past threatened them with Lawsuits... Don't be evil Google died a long time ago, there's nothing to see here, just business as usual.

rlt• 6 years ago

I think we’re starting to see companies discover the limits of how far they’re willing to let employees push their “woke” agenda using the company’s name.

It seems disparaging your own company while ignoring research that counters yours is Google’s limit, but we’ll have to wait to see the research paper if it leaks.

htrp7• 6 years ago

Reading the abstract, it seems to be a pretty vanilla-esque survey paper.... which implies there's some more info that will only come out if this goes to court (doubtful).

samizdis2• 6 years ago

This topic, over the past couple of days, seems to me to have become fraught, polarised and argued to distraction and faction.

It's not a stretch to say that the world has a problem with discrimination, or that big personalities can rock the boat when they go against the grain. Or that corporates have interests to defend.

I'd like in this case, though - not to say that all the other factors aren't worthy of examination as subjects in their own right - to see this paper. The authors/collaborators are leaders in the field. There are legitimate concerns about AI/ML; the validity/reliability of data from which, nowadays, consequential outcomes are derived.

Please, let's ignore - at least for now - the corporate politics, the heat of the race/gender politics, the PR machines weighing in (look at all the news outlets grabbing this atm), and take a look at what the paper says. The right people (those in the field and qualified/able to do so) should be listened to.

As for the politics of this, and the posturing and politicising and agenda-building on all sides: that doesn't help.

AI and ML affect us all now. This is a trend. Let's at least expose the findings of acknowledged leaders in the field - let's see this study - before we descend into the sideshows of politics and factionalism. Please.

Edit to clarify: tl;dr - I've not seen/read this contested paper. Whatever its contents, I want to see them, and imo that is more important than the current controversy blizzard etc.

agentdrtran• 6 years ago

it's absurd to want to wave away stuff like "race/gender politics" when the impact of that "politics" is overwhelmingly obvious in the field.

samizdis2• 6 years ago

Sorry for not being clear. My only point is that I would like to see the research paper. I respect acknowledged and qualified domain expertise. It's an education for people like me, non-academics and not domain experts (but working with some ML/AI stuff). I want to see the thoughts and discoveries and recommendations and cautions and all that - from people who know their stuff.

The rest, to me, is a distraction. This specialist/academic, has a really good rep. I care not about character or friendliness or whatever. As a brain, this person is documented as having the chops. I want to see the report.

danbrooks• 6 years ago

It's standard practice to check in with stakeholders before submitting a paper.

The important issue in the paper seems to be that the paper villainized large language models, which Google has a vested interest in.

The paper was likely publishable with a bit more context in the introduction.

I've been in similar situations and they were handled offline.

The explosiveness of this situation seems to have been prepared the history of Timnit's relationship with Google.

This incident was just the spark.

yodelshady2• 6 years ago

Well, the lesson I'm taking from this is to have a good HR department, and listen to them.

One of their most valuable functions is to stop you "letting go of" people for stupid and petty reasons that blow up in your face, spectacularly.

apta5• 6 years ago

I don't think letting go of a person for causing a tantrum is a bad thing. She threatened to quit, let her own up to it.

throwaway19937• 6 years ago

It's quite common for managers to fire or accept resignations from employees who give ultimatums.

There's also the angry email that she sent to other people ranting about Google's behavior and telling them to stop work.

Either of these scenarios would get you fired any place that I have worked.

goutsuffer• 6 years ago

I am more concerned whether she is really a good researcher or not. If she is, just continue her great work and contribute real good stuff in the field. BTW, what exactly is Ethical AI?

frongpik• 6 years ago

With her credentials, no less than "a famous scientist that's authored landmark papers in AI research", she could easily start her own AI company, or at least a top-tier consultancy, to advance the AI research frontier at the unprecedented pace. She could even hire the top notch researchers from the SJW community and show Jeff what the real science looks like. I'm sure VCs will line up to fund her novel ideas.

Aeolun3• 6 years ago

I just can’t but feel that they entirely miss the point. Nobody is outraged that Google rejected a paper because it needed some adjustments.

What we want to know is if that is actually the case.

idlewords• 6 years ago

This letter reads like Exxon-Mobil chastising their climatologist for not taking into account their latest research on fuel efficiency, or Philip Morris laying into their house doctor for discounting the psychological benefits of rich, mellow flavor.

foobarbazetc• 6 years ago

Jeff tweeted that Timnit “resigned”.

I mean... come on man.

Jeff is way better than this so I really have no idea what’s going on here.

dmitrygr• 6 years ago

Jeff Dean has(/had) a cult following at Google. Internally, there was even a large collection of Jeff Dean jokes (like Chuck Norris jokes of yesteryear).

It is sad to see him drop his credibility and become a PR mouthpiece. I know how this happens and I know why. But it is still sad to watch. Like watching you favourite band sell out to a big label.

geofft5• 6 years ago

Jeff Dean is lying:

https://twitter.com/timnitGebru/status/1335017524937756672

1/Man there’s so much to pick apart. Let’s start with one thing. I want to ask if Jeff Dean has looked at the publication approval policy that he keeps on mentioning in his email. Like, for example, a simple look at the website? Let’s read.
2/First off “Start the PubApprove process at least 1 week in advance of any deadline”. Okay, not sure where the 2 weeks in Jeff’s email came from.
3/ But ALSO “The perfect policy” “There is no such thing as the perfect policy. Fortunately Googlers like to do the right thing. Please do that here—read the policy and do what makes sense.”
4/ ALSO “Meanwhile, we strive to make the PubApprove process as lightweight as possible: hopefully eliminating the temptation to skip it.” I don’t know man you might have to resign immediately if you just “do what makes sense” so beware.
5/ Finally, I wouldn’t want to know what would happen to you if you had “the temptation to skip it” mentioned on the website. Beware researchers
6/In spite of this, we gave a heads up BEFORE even writing the paper—on September 18. Saying that we were about to write this paper. So much to say here, so much. But I’ll stop here for now.

emmelaich• 6 years ago

Key phrase ...

Highlighting risks without pointing out methods for researchers and developers to understand and mitigate those risks misses the mark on helping with these problems

You have to have sympathy for her position but in the end if all you do is offer criticism then it's hard to see you inside Google, let alone as a manager.

darepublic• 6 years ago

Remember that Google fired James Damore for a lot less and the left was really happy about it

bnj6• 6 years ago

My impression was that James Damore was fired when his memo alleged that men were essentially a target of discrimination, while here this researcher it seems has been fired in a way that seems to be an act of discrimination.

I don’t think there’s a contradiction in the reaction, personally.

LocalTrust• 6 years ago

I am reluctant to accuse Jeff Dean of bad faith, but this argument doesn't scan. I've been on the inside of an AI team during a crisis and then seen how senior management spun details to obfuscate critical details to avoid responsibility. Dean is slandering Gebru in a manner that will make it easier to dismiss her work and the work of other AI ethicists (especially women and bipoc) in the future. He and Google are actually themselves guilty of a lacking rigor (i.e. ethical rigor). Worst of all, I expect that racists inside Google and the wider industry will utilize this argumentative structure in the service of neutralizing ethicists and bipoc in the future. This is truly despicable. I used to be a great admirer of Jeff Dean.

BMSmnqXAE4yfe1• 6 years ago

These kind of encounters are going to be more common. AI people want to make AI. But it can be dangerous. And, by definition, we can't control AI.

tinyhouse2• 6 years ago

I read the abstract. The paper doesn't critique Google. It critiques the current focus of AI research of building bigger and bigger models. Google is one of the leaders in building those big models. So what? They also invest in research to make smaller models that perform equally well. They have a huge incentive - those big models are very expensive. I don't think anyone at Google really care if the paper is published. I'm pretty sure it will get published sooner or later. The authors could've addressed the reviewers' comments pretty quickly and then re-submit to another conference. The whole thing blew out of proportion. She picked the wrong battle. Should've bitten the bullet.

About Google's approach to research publication – Jeff Dean

Related Stories