Heard of Idempotency but unsure what it is?

https://timmoth.github.io/AsyncMonolith/posts/idempotency/

by timmoth_j • 2 years ago

29 46 2 years ago

timmoth.github.io

jfengel• 2 years ago

I heard of it once and now every time I remember.

readthenotes12• 2 years ago

I'd make a joke about nullipotency if I could remember what it meant

maname2• 2 years ago

IMO these statements contradict themselves:

Idempotency refers to the ability of an operation to be performed multiple times whilst still yielding the same outcome

and

The issue with the above code is that if the event is processed twice the EmailService will send a duplicate email.

In other words, the operation (three lines of code) yields the same result (sending an email) after repeated execution, and thus is idempotent isnt it? Or is the point that idempotency is harmful in this case?

phoe-krk2• 2 years ago

In other words, the operation (three lines of code) yields the same result (sending an email) after repeated execution, and thus is idempotent isnt it?

The meaning here is: after the operation is performed at least one time, exactly ONE mail is sent. Note the part "at least one time": it can be one, two, or more.

maname2• 2 years ago

thanks, that clicks.

"Simply put, we can perform an idempotent operation multiple times without changing the result.

Furthermore, the operation must not cause any side effects after the first successful execution."

Baeldung got it right.

ReleaseCandidat• 2 years ago

Yes, the first statement can be easily misunderstood. It could also mean that the operation `f` does not have side effects. What it actually wanted to express is that `f(x) == f(f(x))`.

happytoexplain2• 2 years ago

The world is a system that has state. If two invocations of your function with the same parameters both mutate any part of that state, and nothing else mutated that part in between, then it's not idempotent. In this case, there are now two emails in the recipient's inbox.

I.e. "result" refers to state, not actions. "Send one email saying 'hi'" is an action. "There is one email saying 'hi' in their inbox" is a result.

There is also usually an implicit limiting scope (except in very pure functional contexts). E.g. just because two invocations both produce entries in an activity log somewhere doesn't usually count as "not idempotent".

You can also think of it in terms of declarative vs imperative. 'x = 1' is declarative (and idempotent). 'x += 1' is imperative. YMMV with this use of those terms, though.

To oversimplify, an idempotent function is "safe to spam", e.g. if you're not sure it worked the first time, you can do it again without worrying about duplication. A very relevant concept in networked communication.

Edit: I should also clarify that "same parameters" includes any state the function reads that is mutable over the scope in which you would like to define idempotency. I.e. if the function reads some external state that might change between calls, that counts as "parameters" in my simplified definition.

PaulDavisThe1st• 2 years ago

If two invocations of your function with the same parameters both mutate any part of that state, and nothing else mutated that part in between, then it's not idempotent.

Still not accurate. What matters is whether N calls to the function changes the state differently than a single call. If not, then the function is idempotent.

Another issue with idempotency is that there's an implicit temporal component that is rarely explicitly discussed. Most people don't expect a function to necessarily be idempotent over long periods of time.

dragonwriter• 2 years ago

"outcome" here refers to "resulting state of the world", not "actions which are performed as a result of operation".

Which makes it a bad choice of terminology, because while both of those are things for which "outcome" is a natural word choice, the latter is at least as natural of a reading, and is what idempotency is defined in opposition to, so at best the statement is ambiguous on exactly the point it is trying to clarify.

phoe-krk2• 2 years ago

Idempotency: the ability to be able to hit Ctrl+C several times in a row, just to be sure.

erik_seaberg• 2 years ago

Oddly, C-g in Emacs isn't idempotent when it's hung or doing something slow.

https://www.gnu.org/software/emacs/manual/html_node/emacs/Em...

o11c5• 2 years ago

A more subtle case is that C-d (EOF for anyone except DOS) is widely not idempotent. Depending on the exact programs involved, it changes after pressing it about 10 times.

codeulike• 2 years ago

If I've told you once, I've told you a thousand times

readthenotes12• 2 years ago

That's more nullipotency

latentsea3• 2 years ago

George Bush understood idempotency - "Fool me once, shame on... you. Fool me twice... can't get fooled again!"

pudwallabee• 2 years ago

Im not sure why people post these kinds of things. First of all it does not define really anything, it just gropes at the subject trying to find something to hold on to.

And of course theres the crazy web articles that made funny memes about cows multiplying by calling POST over and over.

So Im not going to actually get into any of that text book stuff because alot of it is actually described in academic language, or simply being metaphorical, or describing a “result” as this 5 min blog post did.

So heres what idempotent actually is:

ID - empotent

Means literally “empowered by ID”, or ID aware if that feels better to you. It means that things that have been created will not be recreated because an ID prevents that from happening. And thats all it means. To make that mean something in your app, you have to convert that to a semantic and hopefully contractual interaction.

For example lets use REST.

POST /addresses

{ "first-name": “bob", “last_name": “johnson”, “street”: “foo… }

201 CREATED: { “id”: "8c3d03c4-7fe7-4355-afd4-25e838c4f884" "first-name": “bob", “last_name": “johnson”, “street”: “foo… }

The server returns the ID of the created address, which should now forever identify this address.

So what happens if we do the POST to addresses again and omit the ID? We will be creating a new address we already created. This is exactly what should happen.

But what should happen if we call POST again with the ID?

POST /addresses { “id”: "8c3d03c4-7fe7-4355-afd4-25e838c4f884" "first-name": “bob", “last_name": “johnson”, “street”: “foo… }

You have some options for the response here. Here are some obvious choices:

409 CONFLICT 400 BAD REQUEST

The point being that POST should not process requests to create items that already have an ID assigned as provided by the client.

You can return a response like “Call PUT dummy” but thats not necessary.

And thats really it some shops sort of muddy the waters by saying that IDEMPOTENCY means “upsert”.

Again thats describing a behavior with an intended result not IDEMPOTENCY. While its technically true that APIs should when provided with the same input have the same output it doesnt mean you should code a POST as an upsert.

We have semantics for creates and updates already in REST. So be empowered by ids and make your apps semantics match.

TheAlchemist• 2 years ago

The example is not really great, but Idempotency is a very important concept for everything 'data'.

You very much want to be able to run the same pipeline several times and get the same result, or overwrite the results if you changed the inputs. You don't want to wonder if somewhere down the line, there will be duplicated data that will corrupt results for processes that consumes this data - you just want to be able to re-run your pipeline.

Sadly, that's not how many systems seem to be designed.

o11c5• 2 years ago

This fails to address that fundamental confusion, which is that there are several different (but related) things all called "idempotent":

1. a value `v` such that, for some pure binary function `f` (specified by context), `v === f(v, v)`

2. a pure binary function `g` such that, for all values `x`, `x === g(x, x)`

3. a pure unary function `h` such that, for all values `x`, `h(x) === h(h(x))`. Often this is formed as a partial evaluation of a binary function, as a relaxation of the previous kind.

4. a (non-pure) procedure that, when called with "the same" arguments (FSVO same), has no side-effects after the first time.

5. a binary relation `R` such that, for all values `x, z`, `R(x, z) <-> there exists some y such that R(x, y) AND R(y, z)`

(Edit: note before I clarified my variable names `v` used to be another `x`, and `g` and `h` both used to be another `f`)

dataflow3• 2 years ago

I'm pretty lost, I thought idemopotent was basically a synonym for a projection (which sounds like your #3)... do you have any links to literature using it differently?

GrantMoyer4• 2 years ago

https://en.wikipedia.org/wiki/Idempotence#Definition

GrantMoyer4• 2 years ago

Definition 3 is the same as definition 1. Defn. 3 is idempotence (in the sense of defn. 1) of the function f under function composition. In other words, f ∘ f = f.

BoiledCabbage• 2 years ago

A valuable insight, but doesn't that come with a caveat. Def 3 is only def 1 when the 'x' in def 1 is a function or an operator or some kind. If it's "truly a value" then x is instead an identity value and not idempotent isn't it?

And for #1, what if "f" is a function that selects the first of the two arguments? Then the relation in #1 holds true for all values of 'x', but that doesn't feel like idempotency to me. And we definitely can't say that 'x' is idempotent. As 'x' defined to be "add1" wouldn't be idempotent according to the definition, but would pass the relation if "f" is fst().

I think #1 only holds under compose or 'f's that are similar to compose when 'x' is a function. And at that point you've added so many restrictions you've made definition 3. So I think I disagree with definition #1 as being a def of idempotency.

o11c5• 2 years ago

I've renamed the variables for clarity. So now your comment can be stated as:

  v = h (no longer f)
  f = ∘

But when we're trying to bring clarity to the masses, remember that most people don't think in higher-order functions, so it is certainly necessary to state them separately.

contravariant• 2 years ago

I've yet to encounter the second definition, and the first only when f is associative (otherwise f(x,x) = x doesn't seem strong enough).

All the others are just the many ways x^n can equal x (for all n>0) for various associative operators. Reusing 'f' for both the binary functions and the unary ones is needlessly misleading by the way, the unary f behaves more like x in the other examples.

And yes if you want R^2 = R then the condition only needs to hold for some y. It's related to transitivity.

GrantMoyer4• 2 years ago

You don't need associativity:

(x ⋅ x) ⋅ x

(x) ⋅ x

x ⋅ (x)

x ⋅ (x ⋅ x)

o11c5• 2 years ago

Good point about reusing variables, I've renamed all except the ones introduced locally.

I would only say that these are "isomorphic to" others; in the real world people do think of them differently, so it is important to enumerate them. And in particular people do think "the function", even though different the functions act differently.

(the real question is, did I ever see "left-idempotent" and "right-idempotent" somewhere?)

lxgr• 2 years ago

I’ve heard of idempotency before, so this won’t change my opinion of it.

xwowsersx• 2 years ago

haha good one

jiggawatts• 2 years ago

My mental model is to replace “do” words with “ensure”.

Instead of “send notification message”, it is “ensure recipient is notified.”

wavemode• 2 years ago

Isn't it "idempotence"?

jmull• 2 years ago

Idempotency refers to the ability of an operation to be performed multiple times whilst still yielding the same outcome.

I'm not a fan of this definition because it might give someone the wrong impression.

Suppose the operation is "deposit $10 to account 1234". If that operation is processed twice, a total of $10 should be deposited to account 1234 (for it to be idempotent). By the definition from the article, it sounds like perhaps it would be idempotent if the operation deposited $10 each time it was processed.

stackskipton• 2 years ago

Yea, I see Idempotency as being you can submit same transaction multiple times without duplicate actions. So you to use your example, if I submitted "Deposit $10 to account 1234, transaction ID: ABC" four times, the system wouldn't execute it four times because it has enough data to determine if transaction is duplicate or not.

layer8• 2 years ago

"Same state" instead of "same outcome" would make it clearer.

jmix• 2 years ago

The example is flawed -- if there is an exception incurred before or during the sending of the email, the operation will be erroneously marked as "cancelled" without having completed all of the cancellation flow. Namely, with the example as given, it is possible to have a cancelled order for which an email has not been sent out.

Moving the second line (the one that changes the status to "cancelled") to the end fixes this issue, though it does not implement "exactly once" semantics for the sending of the email. If "exactly once" is desirable, then additional logic is required. But in every case, the example as given is incorrect.

cubefox• 2 years ago

There seem to be at least three meanings of "idempotence":

1. Some x is idempotent relative to f if:

  f(x, x)=x

2. Some f is idempotent if for all x:

  f(x, x)=x

3. Some f is idempotent if for all x:

  f(f(x))=f(x)

latentsea3• 2 years ago

There's a draft standard for how to make POST and PATCH requests idempotent which is quite an interesting read https://datatracker.ietf.org/doc/draft-ietf-httpapi-idempote...

Heard of Idempotency but unsure what it is?

Related Stories