The NeurIPS 2020 broader impacts experiment

https://statmodeling.stat.columbia.edu/2020/12/21/the-neurips-2020-broader-impacts-experiment/

by luu • 6 years ago

58 52 6 years ago

statmodeling.stat.columbia.edu

echelon• 6 years ago

In order to provide a balanced perspective, authors are required to include a statement of the potential broader impact of their work, including its ethical aspects and future societal consequences. Authors should take care to discuss both positive and negative outcomes

Well Google, there you go.

trhway2• 6 years ago

I wonder whether/when "ethical AI" is going to metastasize into other areas of software engineering as well as into sciences, like physics, biology, chemistry. I mean is NN research is more dangerous than say nuclear or is it just self-aggrandizing ?

_tcsi2• 6 years ago

Uh, yes? As just one example, China is already using facial recognition technology to help them identify, control, and jail Uyghurs: https://www.npr.org/2019/12/16/788597818/how-china-is-using-...

throwawaygh• 6 years ago

Nuclear and crypto already have extreme ethics reviews with super high stakes called "export controls" ;)

I kid, but only kind of.

the_snooze2• 6 years ago

In the security research community, it's common for reviewers to expect discussions of things like responsible disclosures, informed consent, data confidentiality, and the legality and ethics of research methods. I think it's a great thing for researchers to be cognizant of their work's broader impacts; not just for harm reduction, but because it makes your work so much more interesting to a wide variety people.

moyix• 6 years ago

This is true – but at the same time the security research community doesn't require that every paper do this, only that if your paper has potential for harm that you discuss how it was managed. It definitely comes up in program committee meetings though (and it tends to provoke some of the most heated discussions).

Permit2• 6 years ago

I'm not familiar the security research community, is it common for papers to be rejected for the reasons you listed?

the_snooze2• 6 years ago

It hasn't been common in the program committees I've served on, and I think that's mainly a function of people generally knowing what the expectations are. I've had my own papers get review comments like "good work, but we need you to include justification for why your approach isn't illegal under the CFAA." Authors (myself included) are generally very receptive to that kind of feedback because it helps keep us out of trouble down the road.

Here's an example of a disclaimer that shows the authors were acting in good faith (not mine):

As performing a security analysis against a running election server would raise a number of unacceptable legal and ethical concerns, we instead chose to perform all of our analyses in a "cleanroom" environment, connecting only to our own servers. Special care was taken to ensure that our static and dynamic analysis techniques could never interfere with Voatz or any related services, and we went through great effort so that nothing was intentionally transmitted to Voatz’s servers.

https://www.usenix.org/system/files/sec20-specter.pdf

linspace2• 6 years ago

Science is objective and there is a long history of morality and ethical changes, and incorrect evaluation to societal impact. I think this is a concesion to political pressure.

marcinzm3• 6 years ago

Science is objective

Nothing done by humans is objective. Objective systems don't, for example, have massive reproducibility problems.

linspace2• 6 years ago

Science cannot be trusted, right?

unishark2• 6 years ago

Nothing done by humans is objective.

This seems like a hyperbole. Obviously there is a lot of garbage that people claim to be science. But 2+2=4 is subjective?

bra-ket• 6 years ago

'Regardless of scientific quality or contribution, a submission may be rejected for ethical considerations, including methods, applications, or data that create or reinforce unfair bias'

In other words science has to pass trough political filter of the day. If reality is wrong, unfair or politically incorrect (in our subjective view), we're going to reject reality.

Prof. Pedro Domingos published a great piece on this https://spectator.us/militant-liberals-politicizing-artifici...

Der_Einzige• 6 years ago

I think your vitriol is wrong, but there is a kernel of truth.

AI with no debiasing is a mirror. It's pattern recognition. Techniques for achieving fair and equitable AI are frequently quite similar to "playing god" and do come dangerously close to implementing "rejecting your reality and substituting my own" into the data through sophisticated techniques for social justice reasons. This can elevate the impact of an AI developers actions far above what they may have expected.

For instance, I am shocked that the right wing media has not reported more about the "censorship" inherent to some kinds of word vector debiasing techniques proposed in papers after the famous "man is to computer programmer as woman is to homemaker?" paper within NLP. Usually the undesirable associations (programmer is "male" and homemaker is "female") are due to a representation gap either inherit to reality (far fewer female programmers) or due to failure to properly write about and represent female programmers in corpra (Wikipedia editors should write more articles about female programmers).

One of the techniques proposed to prevent this bias is to simply find the specific rows in your dataset that maximizes the undesirable association and remove these rows. In practice, most of these "undesirable" rows (let's say a row is a wikipedia article) will have no inherit bias what-so-ever, and only get their poor scores because the article is about someone where the editors/authors wrote a lot about their relationship with their dad (using male words) before talking about their programming career. It leaves me with a bad taste in my mouth to remove not-sexist articles to mitigate or remove sexist associations.

Maybe the OP could have critiqued this kind of work rather than their original post and it may have been more constructive...

free_rms5• 6 years ago

Set aside the details, it's clear that the definition of "ethics" is "things which are currently fashionable in US coastal universities".

The fact that many of these technologies could be used to steer a cruise missile? Barely on the radar (heh). Nothing about poverty, religious issues, the global south.. but something vaguely resonant with US culture wars? Sound the alarm!

Any real conception of "ethics" in my mind should be studiously divorced from fashion and group affiliation.

naresh_xai2• 6 years ago

AI with no dibiasing does not work often enough. I’ve seen enough examples in Computer Vision models to say so. (Even segmentation models tend to rely on external clues and this affects the generalizability of the model). Happy to share more via email.

throwawaysea2• 6 years ago

An interesting aside: the politically charged members of the AI world are trying to cancel Pedro Domingos. For example he was a victim of a (failed) attempt at cancellation by Anima Anandkumar, Director of AI at Nvidia. See what I wrote about it at https://news.ycombinator.com/item?id=25419871

kaitai6• 6 years ago

Oh for heaven's sake. If you're going to be ethically honest about this, note that he tweeted speculation about her porn habits. When is a person allowed to fight back? Is there a sufficiently "polite" way you'd like her to react? Convenient that her side of the argument gets to be labelled "cancellation".

My God. Can we not just have an old-fashioned down & dirty fight these days without it being labelled a "cancellation"?

bdhe• 6 years ago

Setting aside the discussion around the responses on twitter, I think his take on algorithmic bias is too simplistic.

Credit-card scoring algorithms may reject more qualified applicants in order to ensure that the same number of women and men are accepted.

The entire discussion revolves around what defines "more qualified" and why it cannot simply be the output of an algorithm because it reflects biases in training data and even training methodologies. If you just skip past that entire discussion you may think that there is this nefarious worldview of "injecting bias" but that requires a false or at least simplified premise to begin with.

There is a lot of just-world fallacies behind this sort of thinking. The same that leads to suggest that if "X started a business and made a lot of money" X must be by-definition hard working or talented etc. ignoring aspects of generational wealth, opportunities, fallback options if they failed, etc.

Permit2• 6 years ago

Only four papers were rejected because of ethical considerations, after a thorough assessment that included the original technical reviewers, the area chair, the senior area chair and also the program chairs.

Is it possible to read these papers?

CJefferson• 6 years ago

In general the rejected papers are never discussed by conferences, as the authors will want to keep their from being public until they can publish it.

I don't entirely agree with this, and some subjects are moving away from this way of working, but it is still very common in computing.

dhairya• 6 years ago

I agree as well, it's valuable to see rejected papers or at the very least the reviews themselves.

It's interesting that some Neurips workshops are on OpenReview but Neurips itself is not. I've found great value in being able to read the reviews (both as a reviewer and learning how to do good reviews and as writer)

nharada• 6 years ago

I for one am happy to see this.

I often see the argument that technology can be good or bad depending on how it's used, and I would agree with this. However, too often the solution to preventing misuse proposed by people is "don't think about it", which doesn't seem like a good one to me.

Master_Odin• 6 years ago

A lot of these pieces on "ethical AI" just seems like theater to me. I'm sure if you ask the authors of the modern techniques of facial recognition, they'd tell you all about the wonderful and great things you can accomplish with it. And yet we also have state actors using it for great harm, and this is true of a lot of foundational AI work that then powers AI solutions to problems.

Maybe this kind of "push" from conferences will help power a change, but the given the replication crisis (which the author mentions) has been well known for a long time and nothing has really happened on that front, I wouldn't hold my breath.

asciident• 6 years ago

There's a handful of ethical AI tropes I hear over and over. The (made up) story about Target predicting a pregnancy before she knew, the concern of using it to predict parole, face detection not working with dark-skinned people, and the trolley problem. Every talk is some permutation of these. I would love to hear actual solutions, guidelines, and deeper threads of research.

naresh_xai2• 6 years ago

Happy to share a lot more ethical and challenges of non transparent AI via email with you. Bigger companies are pretty much hiding behind non transparent models and trying to ignore their failures at the moment.

tawtawtaw2• 6 years ago

This is a theory-focused academic conference, so all of the work being presented is pretty fundamental, meaning it could underpin all kinds of applications. By analogy, if I invent a hammer that is balanced to drive nails faster, should I consider the broader impact that a bad actor could build a shelter faster?

This is all about politicizing fundamental research, because it's in a field that's in the spotlight. As work becomes more applied, I can see the logic that scrutiny and consideration of its ethical implications should be increased, but pretending that authors of the kind of fundamental advances presented at a research conference like this need to speculate on the kind of end applications they could underpin is just people with nothing of value to contribute trying to muscle in to a popular venue.

kaitai6• 6 years ago

I was a theoretical mathematician for a long time, and even there we had to talk about Broader Impacts for NSF grants, for instance. It wasn't really talking about applications for us, necessarily -- people touted the summer camps they'd present at or the undergrads they'd sponsor for research or the way they would give grant money to help undergrads attend conferences (always tied to the pure math in the grant).

These NeurIPS Broader Impact statements seem a bit different, in that they more explicitly call out ethical considerations.

It's funny though that you ask about hammers & nails. Pure math had a strong tradition of people ending up there because they did not want to do applied math, physics, or engineering that could be weaponized and used to kill people/support oppressive governments/infringe upon basic freedoms. Pure math is not particularly thoughtful about ethics, but many folks ended up there because of their ethical views. The Cold War, World War 2, Vietnam, and French occupation of Algeria had particular ripple effects in pure math because of that.

It would be a shame if ML also became a field people avoided because they didn't want to contribute to evil, in their own view.

Take the longer view: it's easy to think we're the first people ever to think about ethics and science/tech/math, but far from it. How are today's struggles the same or different than Grothendieck's, or Laurent Schwartz's, or Emmy Noether's, or Lagrange's or Kovalevskaya's or...? They all had significant reckonings to contend with.

mkasu2• 6 years ago

It would be a shame if ML also became a field people avoided because they didn't want to contribute to evil, in their own view.

Both ethics and privacy considerations have recently become pretty regular at Computer Vision and Multimedia Processing conferences.

One very popular object detection model (called YOLO) had the main author recently leaving the field because he got concerned about military applications using his research results.

https://medium.com/syncedreview/yolo-creator-says-he-stopped...

uomopertica• 6 years ago

I totally agree. I published one paper at NeurIPS this year and I work in a machine learning subfield where I basically only train networks on the MNIST digits dataset. It felt pretty pointless having to write down that I had no ethical concerns as my study only deals with handwritten digits. Some other researchers in my field got more creative and wrote down how their allegedly more efficient training algorithms would actually stop global warming... that really was pretty pointless.

xiphias22• 6 years ago

While it's great to talk about broader impact of new algorithms, I think it's too late to limit AI from taking control over humanity.

I see exponentially stronger algorithms every year, but the advancement in being able to set a limit to it is nothing so far.

croissants• 6 years ago

I see exponentially stronger algorithms every year

You do? I don't. My sense as somebody near (but not in) the ML research community is that, while there have been recent flashy new things like GPT3 or the new protein folding benchmark, these are more like "solid improvements to applications of things we mostly already know". Are you referring to something else?

xiphias22• 6 years ago

You can take almost any deep learning task that we had 5 years ago, and we can train the same task about 1000x cheaper because of hardware improvements that are slowing down, and algorithmic improvements that don't seem to be slowing down. Jeff Dean had an overview article about it I think.

Also many people forget that we have working self driving cars on the road, 20 years ago nobody thought that it would happen so fast.

lasagnaphil• 6 years ago

The more important issue with ethics in AI is that there aren't that many people well-versed in both technology and philosophy (Throughout history many prominent scientists and engineers also dabbled a lot in the humanities, but today it seems there aren't that much.) Therefore, on one side many people in the humanities greatly misunderstands and either fantasizes or fear-mongers AI up to ridiculous heights (sometimes venturing into science-fiction territory), or many scientists and the engineers think there is absolutely no problem in applying algorithms that can be comparable to phrenology [1] to society.

But most importantly, it seems like there needs to be a fundamental advancement in philosophy (beyond the current state of modernism and postmodernism), if we really want to criticize the negative aspects of technocratic governance. Maybe here's an idea I had for a while: I think there is a deeper lineage to machine learning that nobody is talking about: optimization. (Hot take: isn't what we're all doing with deep learning just optimization to fit large datasets?) I think in order to actually create a meaningful discussion about today's AI, it is crucial to look back at the history of optimization (all the way back into the 30s Soviet, where they tried to use linear programming to their planned economy and failed [2].) I think there needs to be a philosophical framework capable of articulating the effects of optimization technology on our society, before we can go any further and tackle the issues with today's and tomorrow's AI.

[1] https://en.wikipedia.org/wiki/Phrenology [2] https://crookedtimber.org/2012/05/30/in-soviet-union-optimiz...

Fede_V• 6 years ago

I'd rather have a section of the conference that's clearly dedicated to ethical issues in AI - or - mandating rigorous discussion in papers where it is clearly warranted (ie, anything concerning facial recognition/generation, de-identification, etc) than asking all authors to write an additional section.

NeurIPS is an extraordinarily competitive conference. Most authors will look at it as another way to potentially be rejected by a reviewer/area chair (in spite of all assurances about the process), and will write something incredibly trite and bland to make sure nobody is upset.

Another concern is that this leads to a death by a thousand cuts - it's very easy to justify asking authors to write an additional section - but - if someone with sufficient authority doesn't say no, the logical endpoint is something like the application process for tenure track positions, where you need to produce something in the order of hundreds of pages (teaching statement, research statement, cover letter, diversity statement, list of funding, recommendation letters, etc).

throwawaysea2• 6 years ago

The injection of politics and fuzzy notions of morality into science and technology is corrupting the objective truth seeking foundations of these fields. NeurIPS needs to abandon this direction immediately or else its legitimacy will be in question. This will lead to the publishing authors affixing pandering impact statements to cater to the personal politics whims of those judging their work, and will also lead them to avoid risky exploration or challenges to the political status quo. The legitimacy of AI Ethics as a subject is in serious doubt already after the Timnit Gebru fiasco, but if NeurIPS wants to invite politics into the AI world, they should invite those writing papers on ethics to analyze others’ work instead of requiring such impact statements of authors who are focused on core technology.

The NeurIPS 2020 broader impacts experiment

Related Stories