Formal Models for Ethics

https://cartesia.link/2020/09/28/the-ethical-question-mk-ii/

by queueue • 6 years ago

46 60 6 years ago

logicchains12• 6 years ago

Given the recent push towards formally verifying mathematics, it's surprising how comparatively little effort's been put into formalising ethics. Fundamentally an ethical system is not that different from a mathematical one: there are some assumptions, some deductive rules, and a potentially infinite set of conclusions that result from applying those deductive rules to those assumptions. One particularly important property to verify about such a system is consistency: if it can prove conclusion "X", it should not also be able to prove conclusion "not X". An inconsistent system can prove anything, so is essentially useless for reasoning.

m12k• 6 years ago

Philosophers have been trying and failing for millennia to come up with hard rules for ethics. A rule that is essential in one context invariably leads to absurdity in others (there are many examples involving people strapped to train tracks). So an attempt at formalizing this will not just be ambiguous, but likely unsound and contradictory. While mathematics seem to be this perfect structure encoded in the fabric of the universe, just waiting for us to discover and describe it, ethics in contrast seems to be a very fuzzy, imperfect, ad-hoc creation of mankind.

logicchains12• 6 years ago

Philosophers have been trying and failing for millennia to come up with hard rules for ethics.

They've been trying to find one true/correct system of ethics, which is doomed to fail due to https://en.wikipedia.org/wiki/Regress_argument. What I'm talking about is not trying to find new systems, rather just formalising existing systems, so that they can be analysed more rigorously and checked for consistency. Much as for instance one can formalise and compare homotopy type theory and set theory as foundations for real analysis.

RivieraKid3• 6 years ago

Yeah, ethics is subjective and it's fascinating that many smart people don't realize this. Ultimately it always boils down to "I feel this is right".

Of course, you can have objective claims such as "if all people followed this principle, it would have this outcome".

tr3523• 6 years ago

There has actually been quite some work since the 50ies up to the present day on deontic logic, which is reasoning about ethical concepts such as obligation and permission.

netcan4• 6 years ago

There has always been a ton of effort at formalising ethics, political theories and philosophy generally in a mathematical-like way. The whole of modern philosophy, especially political philosophy , is a product of that effort. That's basically what the enlightenment was.

The problem, IMO, is that formalistic, math-like approach produces "an inconsistent system can prove anything, so is essentially useless for reasoning." Going for fundamental, objective truths with a mathematical standard as an ideal can go one of three ways: delusion, nihilism or sociopathy.

Peter Singer, Jeremy Bentham and other utilitarians came up with a "moral math" system that (a) sounded morally reasonable at first pass and (b) are theoretical applicable widely. The problem they encounter is extreme conclusions. Bentham was responsible for some horrific prison/reformatory systems. Singer ends up with some sociopathic conclusions like morally good infanticide.

Ayn Rand went down a philosophical worm-whole trying to tie her philosophical system into one "theory-of-everything" bow where pure (Aristotelian) logic yields conclusive answers to all epistemological, moral, political & economic questions. This certainty, rather than just the extreme or counterintuitive (selfishness as a virtue) elements of objectivism is what gives it that cult-like flavour.

Ayn Rand was simply following the frame created by Marxism, and Trotsky particularly. Marxism of that era was all about airtight rationality. Communism wasn't a choice, but a dictate of pure reason. Marxists got this idea from French revolutionary philosophers, who started with the idea of "rationalising" everything with math-like systems of thought.

What all these have in common is a desire to create irreproachable systems. Like math, like physics. In practice, they created religions. Religions are irreproachable too. The way religions become irreproachable is by banishing heretics. That's what the "secular" rationalist philosophers did to. Rand was famous for her banishments.

If you are after scientifically valid, reductive, Popperian insights only... you might just have to accept that this is an amoral viewpoint. Morals just don't exist in the mathematical realm.

logicchains12• 6 years ago

What all these have in common is a desire to create irreproachable systems. Like math, like physics. In practice, they created religions. Religions are irreproachable too. The way religions become irreproachable is by banishing heretics. That's what the "secular" rationalist philosophers did to. Rand was famous for her banishments.

I'm not talking about using mathematical, logical approach to find the "most correct" philosophy. As Wittegenstein noted in Tractatus, the "most correct" approach is: "To say nothing except what can be said, i.e. the propositions of natural science, i.e. something that has nothing to do with philosophy: and then always, when someone else wished to say something metaphysical, to demonstrate to him that he had given no meaning to certain signs in his propositions".

Rather I'm talking about using formal methods to analyse existing systems. Identify their axioms, identify the deductive rules they permit, and determine how consistent they are. Maybe even formally verify this in a proof assistant. Then it'd be also possible to e.g. explore what different systems can and cannot prove (somewhat akin to https://en.wikipedia.org/wiki/Reverse_mathematics). An example kinda along these lines is Lawvere's work on modelling Hegelian philosophy as modal type theory: https://ncatlab.org/nlab/show/Hegel%27s+%22Logic%22+as+Modal....

ahelwer• 6 years ago

If we want to be very reductive about it there is not a great reason to believe we can use formal logic to inform the chosen(?) actions of huge aggregates of molecules jiggling around. All ethical systems that have withstood the test of time (the gold & silver rules) use subjectivity. Everything else falls prey to absurdity.

I recall an objection was raised (possibly by Alasdair MacIntyre? it's been a while since I got wrapped up in all this) that any system of ethics must be inherently self-contradictory. The reason for this is something like any time we spend in pursuit of correcting or preventing some moral wrong is time not given doing the same for another. And there are certainly a great many moral wrongs going on according to any given system of ethics - probably all of equal importance.

Separately we might also ask whether we even want to solve the problem of ethics. Imagine tomorrow we develop an ironclad formal moral system - everyone is forced to agree it is correct through sheer reason alone, and it meaningfully informs our every action. Would you really want to live in such a world? It seems a world without freedom, or one which would quickly become that way. All-encompassing totalitarianism. [0]

In other threads you say you aren't interested in proving which system of ethics is most applicable to human endeavours or anything like that. At this point we get into "higher order truths about chmess" territory, as in - why bother? [1]

[0] https://www.existentialcomics.com/comic/139

[1] https://ase.tufts.edu/cogstud/dennett/papers/chmess.pdf

logicchains12• 6 years ago

Separately we might also ask whether we even want to solve the problem of ethics.

The article's author presents a good example: for AI ethics. If we want to prove "an AI following this system won't kill all humans (or has e.g. less than 0.1% chance of killing all humans)", we need to be able to formally reason about that system.

In other threads you say you aren't interested in proving which system of ethics is most applicable to human endeavours or anything like that. At this point we get into "higher order truths about chmess" territory, as in - why bother? [1]

By that logic, why bother with any of the higher, more abstract fields of mathematics?

pron• 6 years ago

Have you heard of paraconsistent logic?

Ethics does not aim to practically tell us what we should do at all times, but to give us the tools to think about such question. A system of ethics where two opposing conclusions can be reach could be perfectly fine.

logicchains12• 6 years ago

In paraconsistent logics one cannot use a proof of "X" and "not X" to prove any arbitrary statement, so it can indeed still be useful. However we still need to formally reason about the system to be able to establish that it is indeed a paraconsistent logic (that the principle of explosion doesn't apply).

asdasdasdas5453• 6 years ago

There has been at least since 1951 since the famous publication of Arrow's impossibility theorem.

Extract from Wikipedia :

"The framework for Arrow's theorem assumes that we need to extract a preference order on a given set of options (outcomes). Each individual in the society (or equivalently, each decision criterion) gives a particular order of preferences on the set of outcomes. We are searching for a ranked voting electoral system, called a social welfare function (preference aggregation rule), which transforms the set of preferences (profile of preferences) into a single global societal preference order. Arrow's theorem says that if the decision-making body has at least two members and at least three options to decide among, then it is impossible to design a social welfare function that satisfies all these conditions (assumed to be a reasonable requirement of a fair electoral system) at once: [...]"

https://en.wikipedia.org/wiki/Arrow%27s_impossibility_theore...

coldtea2• 6 years ago

Fundamentally an ethical system is not that different from a mathematical one: there are some assumptions, some deductive rules, and a potentially infinite set of conclusions that result from applying those deductive rules to those assumptions.

Even simple enough math starting from formal axioms are incomplete (Godel) [1]. Real world ethics, starting by ethical assumptions will be 1000 times as full of holes and inconsistencies. And that's before we even consider language ambiguities in the much more complex "ethical assumptions".

[1] "The first incompleteness theorem states that no consistent system of axioms whose theorems can be listed by an effective procedure (i.e., an algorithm) is capable of proving all truths about the arithmetic of natural numbers. For any such consistent formal system, there will always be statements about natural numbers that are true, but that are unprovable within the system. The second incompleteness theorem, an extension of the first, shows that the system cannot demonstrate its own consistency".

logicchains12• 6 years ago

Even simple enough math starting from formal axioms are incomplete (Godel) [1]. Real world ethics, starting by ethical assumptions will be 1000 times as full of holes and inconsistencies.

Just because a mathematical system like set theory is incomplete (can't prove its own consistency), that doesn't mean it's "full of holes and inconsistencies". And just because a system cannot prove itself consistent, doesn't mean it can't be proven consistent in some larger system (for instance how we can use ZFC to prove that Peano arithmetic is consistent).

RcouF1uZ4gsC• 6 years ago

I think ethics is an interesting study. However, using ethics to try to guide behavior is a waste of time. Basically, every ethical system has axioms. By changing your axioms, you can get your ethical system to justify or condemn any behavior you want.

So people end up doing the same thing, they just have more "logical" explanations of why their behavior is justified.

guerrilla3• 6 years ago

The axioms are however not arbitrary. Philosophers spend most of their time arguing about which sets of axioms and specific axioms are better than others. You should read up on it. They take great care in trying to avoid the pitfalls you're talking about.

You-Are-Right• 6 years ago

where?

PeterisP• 6 years ago

That's kind of the point - philosophers spend most of their time arguing about which sets of axioms and specific axioms are better than others, there are many very different sets of axioms with very different conclusions and properties, and the choice of these axioms is essentially arbitrary, allowing you to get to pretty much any arbitrary conclusions.

You'd be able to say "The axioms are however not arbitrary" when (if!) philosophers have stopped arguing and the field of philosophy can say "for ethical axiom sets A-E, while having lots of earlier philosophical discussion, there's now a strong consensus in our field that they are wrong and flawed because of reasons, for any practical applications please ignore them and use axiom set F as that is the better one that addresses the known flaws in others". Currently, there's lot of philosophical discussion on ethics, but little of solid decisions and conclusions from these discussions; all the discussion is still at the stage of increasing ambiguity and introducing hypotheses instead of increasing our knowledge about reality by eliminating and discarding most of these hypotheses.

exdsq3• 6 years ago

I’m starting a one year research project on formalising ethics at Cambridge this month. The one big issue that stands out, to me, is how we source ethics. Until we solve that, I don’t really see how we can practically apply any of these things.

throwaway_pdp093• 6 years ago

What does it mean to 'source ethics'. I don't understand the phrase.

exdsq3• 6 years ago

Say we have a system of implementing some ethical framework in an AI application, how do we actually decide on what those ethics are. The problem gets harder the more global the application is - ethical values held in one place won’t be the same as another.

anticensor3• 6 years ago

What about a self-stabilising globally utilitarian (towards maximal societal output and overall comfort in expense of individual pleasure) system, similar to what we have with TCP (which is designed to maximise global packet flow and reliability in expense of local slowdowns)?

joppy• 6 years ago

What does this mean though? By utilitarian you mean to optimise for some utility function: how do you pick it? Does it allow for 99% of people to be fine while 1% live in absolute squalor? Does maximising societal output mean draining natural resources dry for generations to come? How does this treat personal autonomy, which is a way that many people derive meaning in their lives? Why should something as complicated as human ethics be reducible to terms as simple as a networking problem?

hliyan• 6 years ago

As of late, I explain ethics to myself as:

"A system that governs the behavior of a conscious agent that experiences the qualia of pain, within the same causal domain as other such agents, to arrive at the optimal dynamic equilibrium between minimizing individual pain and average group pain."

It's how I would formulate Kant's Categorical Imperative.

throwawaybutwhy• 6 years ago

What have I just read? A thinly-veiled ad for an MIT Press book? A jumble of rambling thoughts?

The other possibility, of course, is being too dense to comprehend the text. Can't rule it out.

JohnStrangeII2• 6 years ago

I'm working in formal ethics, mostly on formal aspects of value structure, and agree with you. This article is a hodgepodge of many different ideas, some of them interesting, others a bit naive, and has almost nothing to do with formal models for ethics. There is plenty of research in formal ethics such as deontic logics, abstract argumentation frameworks for preferences and norms, nonmonotonic logics for value-based reasoning, normative systems, input/output logics and formal axiology.

One thing that the author does not address in enough detail is the simple fact that there are many different ethical traditions that come to different conclusions about particular normative issues, and that there are plenty of authors in ethics who (still) consider their business a normative one. A "bottom up" machine-learning based approach to this would invariably fail and miss the whole point. There are some ethicists who consider it mostly a descriptive endeavor - Schopenhauer was one of the first, for example -, but they are in a minority.

As long as experts in ethics cannot agree what the "right" ethics is, it's hard to see how we would be able to teach it to machines. Many meta-ethicists including me would even deny that there can be an "expert" about moral and particularly about ethical questions at all. However, I have no doubts that various robotics companies will implement those ethical rules and approaches that best serves their interests as companies.

That's why I think robot ethics is kind of misguided. What we need is laws that regulate AI and put its use into a legal framework and closes loopholes. This is a political, not just an ethical issue.

renee_sanscarte2• 6 years ago

Hello, I saw the pingback for this thread today. I'm surprised (and a bit giddy) that my obscure little blog wound up on HN, so I hope you don't mind my coming in a little late.

The purpose of this post was more exploratory, rather than for the sake of drawing specific conclusions, which I am aware can be frustrating. Thanks for taking the time to read it anyway.

You are correct that I was not very interested in formalizing ethics. I'm not so sure why OP chose to use that to title the thread, because only a part of the post was oriented towards overviewing the AI safety discourse, and within that discourse, the development of a complete formal system for ethical reasoning is considered a naive way to attack the problem. I never wrote about the formalization angle directly.

The problem, such as it is, is that it's troublesome to encode what we want of an AI agent, even more so to encode how it should reason about what we want of it under unforeseen circumstances, no matter what it is that we want. Ultimately, in order to successfully implement regulation, an agent must be "taught" (in whatever sense) to make judgments according to what is wanted of it. Technically speaking, this is a hard problem. However, there does not necessarily need to be an absolute answer as to what the correct ethics is, in order to teach a machine methods of reasoning about ethical problems, nor for us to reason about what sorts of imperatives a machine should be reasoning about.

The thrust of that section, however, was to raise an issue with the (in my opinion) narrow way in which that problem is typically formulated, to which you have also alluded. Specifically, my beef was ontological (which, for the sake of communication, was distilled into the bottom-up/top-down distinction), but there are conceivably other gaps.

This could have been elaborated; ontological approach is one of the ways in which ethical traditions are distinguished, but far from the only one. Any distinguishing factor may have been missed by an approach like IAD.

That's why I think robot ethics is kind of misguided. What we need is laws that regulate AI and put its use into a legal framework and closes loopholes. This is a political, not just an ethical issue.

This much I agree with wholeheartedly. I made the case that we shouldn't shaft the ethics for the politics, but the same goes for the reverse. Where how we consider ethical problems comes into play again in the second case, is that putting the use of AI into a legal framework and "closing loopholes" has roughly the same shape as the problem of AI ethics, it just chooses potentially different parties to determine what it is we want of the agents.

Anyway, thanks again! If you have a moment, I'd be interested in reading what you thought was naive in my approach.

tunesmith• 6 years ago

I enjoyed reading Formal Ethics, a textbook by Harry J. Gensler. The basic gist is that he formalizes the Golden Rule from some simpler axioms. From my error-prone memory, the four axioms were along the lines of: 1. If you ought to do something, then do it (aligning belief with action), 2. If something ought to be done, then a relevantly identical something also ought to be done (universalizability), 3. One ought to think and live consistently with logic (rationality), and 4. If one is in favor of an outcome, one should be in favor of the outcome's prerequisites (ends and means).

From that you get to concepts like consistency, conscientiousness, impartiality, and eventually the stricter form of the golden rule (which encompasses both "do unto others what you'd have done to you" and "don't do unto others what you wouldn't have done to you").

There's a cool looking symbolic logic version of it, too. My wife and I put it on our wedding poster. :)

Anyway, to echo other commenters here - it's not so hard to come up with ethical axioms (opinions/values) that are widely shared. The challenge is in keeping the reasoning consistent after that point.

Formal Models for Ethics

Related Stories