What Is JSON Patch?

http://jsonpatch.com/

by tsujp • 4 years ago

148 169 4 years ago

jsonpatch.com

coolgoose• 4 years ago

It is a really verbose spec.

Why not just send partial fields and handle it server side?

Not to mention its more data over the wire for small changes.

rendall5• 4 years ago

Why not just send partial fields...?

You have:

  {
    "baz": "qux",
    "foo": "bar"
  }

Then you send the patch:

  {
    "baz": "boo"
  }

It's clear you want to change baz value to boo, but did you mean to delete foo or just ignore it? If "ignore", how do you indicate delete? If "delete", you will now have to send the entire JSON every time, anyway, so it's not a patch. Whatever your answers, there, it's not a "why not just..." situation.

You have a deeply nested property in a gigantic JSON document that you want to move, "why not just" send

  { "op": "move", "from": "/a/b/c/d", "path": "/a/b/c/e/f" }

To be clear, I have never seen JSON Patch before, either.

gregorty2• 4 years ago

Sending it with different HTTP verbs might do the trick. For your example:

      {
         "baz": "boo"
      }

DELETE would remove baz, PATCH would change baz, and PUT would, in effect, remove foo.

jayd164• 4 years ago

Well you could send null to delete a field. { "foo": null }

Append, insert at, remove, and remove at are more ambiguous.

crabmusket2• 4 years ago

This exists, and it is called JSON Merge Patch.

https://www.rfc-editor.org/rfc/rfc7396.html

danw1979• 4 years ago

I know the first example in the docs is intentionally short and simple but it did make me laugh that the patch was at least double the byte count of the thing it was patching.

rendall5• 4 years ago

That's like laughing that a program for a text editor is longer than a text file; or a git history for a file is longer than the file.

danielheath• 4 years ago

I mean, it follows immediately after a bit praising the transfer size savings. That’s at least worth a smirk.

salil999• 4 years ago

Where would this be useful? I am not trying to bring the article down but I'm genuinely interested in knowing where/how people would use this. It seems like just locally modifying a payload would be more efficient?

Hamuko• 4 years ago

Kustomize patches.

mathgladiator2• 4 years ago

JSON merge ( https://datatracker.ietf.org/doc/html/rfc7386 ) is the entire basis for how I persist document changes to disk and to clients ( http://www.adama-platform.com/2021/03/28/json.html ).

Arrays require some care to handle which is why I'm got some extensions in my client, but a super power that JSON patching enables is that it can be algebraic in nature. That it, we can define a patching which is associative (another extension where you preserve nulls) such that (X + da) + db = X + (da + db) = X + dc where da + db + dc

This allows you to leverage flow control when congestion happens (especially on mobile connections).

nigma1337• 4 years ago

Using it for k8s currently, in a request like

[{"op": "replace", "path":"/spec/template/metadata/annotations/restarted-at", "value": "'$(date -Iseconds)'"}]

to restart/redeploy.

kgeist• 4 years ago

I can think of realtime collaborative document editing - patches are sent to the server instead of sending the whole JSON-based document every time and the app resolves potential conflicts.

chii3• 4 years ago

But i wonder if you have to use a very long pointer expression to identify a single character being typed, you'd end up with much higher overhead.

EdwardDiego• 4 years ago

I've used it with K8s and kustomize a fair bit.

jurple• 4 years ago

We have a supplier who uses it to distribute daily updates to their (large) product catalog. It helps that most of the mutations are to specific fields like availability and price, while EAN/GTIN etc tend to be stable. Deletions are trivial as well.

coredog64• 4 years ago

Amazon uses this for changes to API Gateway. Take that as either a positive or a negative :)

wvh3• 4 years ago

Software such as Kubernetes uses it with different processes writing to the same data fields, but different subsets. If you have a typical API with only one main blob that gets updated in its entirety, you might not avoid conflicts using the PATCH approach.

YakBizzarro• 4 years ago

At first it looked interesting, but as pointed out by other people, it's enough to have a 'partial' json with the field that need update. The syntax stays the same, just add what you want to update. We do exactly that and it's very easy

zdragnar• 4 years ago

The advantage is in mutating arrays. You don't need to send the entire array back, just the commands for which index gets added/ removed / replaced.

Also, there's no way to differentiate between deleting a key from an object and leaving it unchanged if you send a partial patch, assuming that `null` is a distinct value different than actually deleting the key itself.

api_or_ipa• 4 years ago

To your second point, the JSON spec allows `null` but not `undefined`, so most js serializers (all?) don't send (or throw an exception) for any `undefined` from any object you attempt to send. To delete a property, you would send the entire surrounding object. Or you can simply define your data-layer without transient key/values. It's certainly not a necessary feature to have.

weird-eye-issue• 4 years ago

How do you delete fields? What about complex documents with dozens of nested fields?

For really simple use cases you don't need this but I've used something similar for an application where the JSON documents could get in the megabytes and it was a lifesaver.

mathgladiator2• 4 years ago

A comparison to https://datatracker.ietf.org/doc/html/rfc7386 would be nice. My gut is that the patch was born due to the copy aspect as well as the awkwardness of arrays.

I got around arrays by simply not needing them for the most part.

rendall5• 4 years ago

This comparison is in the RFC the article links to.

allenu• 4 years ago

I actually have a need for something like this. I wrote a single-user flashcard app which supports multi-device sync. The way I sync is having each device write to its own journal file a list of commands, such as "create card" or "update card with ID 1234" or "insert card evaluation". To "sync", each device just pushes up its journal to a common file store and pulls down each other device's journals. Then it just plays back all the edits in timestamp order. (It works fine for single-user since you're very unlikely to be using two devices at the exact same time.)

Anyway, I never did end up implementing an optimal "update card with ID 1234". If you edit a flashcard's front text, instead of updating the front text field alone, I ended up just writing out an entirely new copy of the flashcard in the journal (with the ID 1234 encoded). Super inefficient.

I was toying with the idea of coming up with a simpler format that does basically what this JSON patch does, where you an just encode "update object 1234's field foo.bar.baz to string ABC".

I figured I'd only need a handful of operations, to insert a new object, replace a node in an object's property tree given a key path, delete an object, append to an array property, etc.

If anyone has other solutions other than JSON Patch, I'd love to hear about them.

rendall5• 4 years ago

JSON Patch seems like a good solution.

samwillis4• 4 years ago

You should also check out Yjs/Yrs or Automerge which implement merging data structures using CRDTs. Most people think of them as useful for colabrotavive real-time editing from multiple users but they are just as good at offline merges. They are perfect for situation like you describe where you have one user with multiple devices.

Apple Notes use’s CRDTs internally for exactly this.

Last year I experimented with an app architecture that used CouchDB/PouchDB for for synchronising data for a single user, multi device app. Then using Yjs to merge the conflicting edits - it worked incredible well.

If I had the time I would love to build a Yjs/CRDT native CouchDB like database that could use the Yjs state vectors as a wire protocol for syncing…

dasz• 4 years ago

If there's some more depth or otherwise more detail then you should consider writing an article or blog post. Sounds interesting.

LukeEF• 4 years ago

There is a diff and patch API described here - the underlying database (TerminusDB) also has the concept of merging records, which could fit to your use case: https://terminusdb.com/docs/index/json-diff-and-patch

ToJans• 4 years ago

It's the logical next step in the evolution of JSON:

- XML/XSLT

- JSON/JSONPatch

pastage• 4 years ago

XSLT can do too much, you can even draw a map with XSLT with styles like Osmarender[1]. Javascript is already the XSLT of json and it is good enough at giving alternatives to a locked down format.

[1] https://wiki.openstreetmap.org/wiki/Osmarender

SemanticStrengh• 4 years ago

And xpath

ajuc• 4 years ago

Escaping is weird. Why ~0 and ~1 instead of \ and \\ ? Also

Finally, if you need to refer to the end of an array you can use - instead of an index. For example, to refer to the end of the array of biscuits above you would use /biscuits/-

how do you refer to "this value" in this example?

    {
       "biscuits": {
           "-": "this value"
           "zzzz": "something else"
       }
    }

masklinn4• 4 years ago

Escaping is weird. Why ~0 and ~1 instead of \ and \\ ?

Possibly because JSON already has them, so you need to further escape the escapes, which is difficult to author and read?

hombre_fatal2• 4 years ago

"/biscuits/-" addresses "this value" because biscuits is not an array.

If you check out the implementations at the bottom of the post, you'll see that, at runtime, if biscuits is an object they index into `obj["-"]`. And if it's an array, they special-case "-" and index into `array[length]`.

chii3• 4 years ago

so the spec has implementation specific behavior that's not fully specified by just the spec?

i rather see they implement jsonpath - https://tools.ietf.org/id/draft-goessner-dispatch-jsonpath-0... instead of inventing their own xpath-like expression language tbh.

Edit: i suppose i shouldn't be surprised there's multiple standards to varying degrees of acceptance: https://datatracker.ietf.org/doc/html/rfc6901 so may be it is fine to use this expression language...

jayd164• 4 years ago

Seems a bit treacherous that it doesn't differentiate numeric keys and array indices. Why not use JsonPath, I wonder?

toomim4• 4 years ago

In addition to the other limitations, this doesn't let you edit strings. You can't insert text into a string, or delete text. This makes it useless for collaborative editing.

mlajtos• 4 years ago

JSON Patch is not designed to be a solution for collaborative editing. If you need something for that, check out Yjs and Automerge.

toomim4• 4 years ago

Yjs and Automerge are not patch formats. They are CRDT libraries. You are comparing apples and oranges. If you want to send Yjs or Automerge patches over the network, you still need a patch format.

forty• 4 years ago

I don't know if there are good use cases for JSON patch, but I found it pretty inconvenient to use when implementing a SCIM server (which uses JSON patch for PATCH method updates)

madmax108• 4 years ago

I feel like nearly every team which has worked with JSON for a certain number of projects automatically starts building out their own "spec" for how to handle additions/deletions/updations to JSON instances on-the-fly.

I've built atleast 3 different specs for what could be described at JSON-Patch for different projects, and each has a slightly different semantic. eg. in one implementation someone suggested in another comment, instead of explicitly passing an "operation", I just pass "key": null and it gets interpreted as delete everything below this key. In another case "key": null and "key" not existing are treated as different cases etc.

TBH I'm glad that there's some form of standardization for this, but I wish it was more publicized so folks would stop building their own specs for this.

giancarlostoro2• 4 years ago

Am I the weird one out, I've always just build plain old CRUD applications, never really had to PATH JSON, I have some alternative ideas, like just sending the lastUpdated value and ID's to the back-end and getting back any records that are far more up to date, but I wouldn't even bother diffing it, unless there's a good strategy for that, aside from caching it or some fancy database feature I've never heard of.

Genuinely curious how you troubleshoot JSON-Patch it feels like something that adds more mental overhead than it would be worth if it breaks. I understand how it would work in practice from a front-end perspective, I'm unsure about the backend though.

My issue is, if I know what's changed I rather just send the ID's and values that changed to the front-end at that stage, or just those records respectively.

wvh3• 4 years ago

Patch and variations are popular in software such as Kubernetes, where multiple actors are writing to the same field sets. In distributed or highly concurrent environments it's quite likely to get a conflict on update, but in those situations where actors have interest in different subsets of the data, touching only those fields you "own" can avoid the read-update-conflict retry cycle.

tyingq• 4 years ago

It is sort of fun to watch the cycle where json replaced xml mostly because it was simpler. Then to see things that look like xslt, xpath, etc, come back around.

Spivak2• 4 years ago

The tragic flaw of XML was that there was xml.parse equivalent that could take a generic XML document and map it to actually useful types. I guess you could get a Tree<XMLNode> but that's not really all that useful when people want arrays, numbers, hashes, and strings. XML could have been amazing with a common schema for "basic types" with a clear "next step" for structs.

HWR_143• 4 years ago

I agree. It's one of those "why am I paying for all theses features no one uses" situations. An then everyone poorly reimplements the 10-15% of features they were using. And there are no standards for those 10-15%.

Meanwhile, I'm actually not aware of any XML patch standard.

wvh3• 4 years ago

That's true, but XML tends to map rather poorly to basic data types in common programming languages. With JSON you usually get to work with native objects or map structures; with XML, you're stuck with either tree or event APIs, which are a lot more complex and add another layer of indirection.

ellisv2• 4 years ago

Yup. We implemented our own thing a few companies ago. It wasn't great but we didn't know about JSON Patch back then. It would have saved us a lot of time and pain.

petkeviciuz• 4 years ago

Very useful within a business I'm working with. There are multiple data providers who's data shall fall into a single domain data model and they are providing that data in vast amounts. We deal with that by normalizing data, comparing it to present state of data and enqueueing patches into a queue from which application aggregating that data is picking and applying updates. Maybe one small thing regarding Java libs - jsonpatch comes under double license and is not suitable for commercial use so beware, better use zjsonpatch (I also like it more because it's more intuitive).

beeforpork• 4 years ago

Why are pointers strings? They are really multi-indexes, so why are they not arrays of strings (for hashes/"objects") and numbers (for arrays)? This would have avoided the need for escaping and for string operations in general, which complicate the whole thing.

formerly_proven• 4 years ago

Quite honestly, I've yet to see an RFC with JSON in the title that isn't a disaster in one way or another.

laurent123456• 4 years ago

I guess it's more user friendly to have the path as a string, except of course most of these patches will be machine generated and processed, so the user friendliness is a bit pointless.

throwawaymaths• 4 years ago

Shorter encoding. Count the bytes:

    ["foo","bar",1,"baz","quux"]

    "/foo/bar/1/baz/quux"

It's also less parsing, and for many systems a less wieldy/less errorprone entity to pass around your runtime. You unpack it at the time you need to dereference the JSON pointer.

jitl• 4 years ago

I don’t get this either, the byte savings are trivial and escaping/unescaping logic adds needless encoding complexity to a concept that otherwise does not need string encoders!

At Notion, we use Array<string | number> as you suggest.

anentropic2• 4 years ago

partly probably just because JSON Pointer spec already exists

it's used in OpenAPI specs

apparently the goal was something that could be transferred in string contexts like a URI fragment, as well as JSON https://datatracker.ietf.org/doc/html/rfc6901#section-1

simiones• 4 years ago

I think it's natural to do it this way, since it's common for a REST API to also support GETs inside objects using URLs. That is, `GET /` can return `{"top": [{"key": "value"}]}`, while `GET /top/1` can return `{"key": "value"}`. So, reusing the same syntax to refer to fields comes very naturally if you already have this support.

viach• 4 years ago

If you use JSON for configuration (as it should be), why not update the whole file and get change tracked in VCS? If you use it as a database, why not Mongo then?

oneeyedpigeon• 4 years ago

This approaches captures more semantic info. about what's actually changed than changes at the line/character level do.

ellisv2• 4 years ago

I've used JSON Patch via Kustomize to perform last mile changes for k8s manifests. For example, add `label=dev` to everything in this chart. Much easier to manage differences across the environments.

johnorourke2• 4 years ago

Use case example: in a strategy game, we're implementing this format so that each weapon, skill, terrain, and unit has the chance to add patch commands during a specific part of the turn, which are then applied to a document representing the entire state of a battle. This makes it easier to handle conflicting effects and changes etc.

GLGirty• 4 years ago

How do you deal with the fact that the patch operations are not associative? This seems like a recipe for conflicting effects... you don't want the result to depend on the order you picked up items. Do you batch the effects? (e.g. deletes, then inserts, then modifications?)

What is the behavior of your chosen library--mutable or immutable operations? I.e. do you pay the penalty of making a deeply nested copy of your scene graph for each turn?

Asking for a friend....

johnorourke2• 4 years ago

Great question! We chose to make the order explicit and important - for example if we have two battling units, one with "unit reduces counter attack strength by 50%" and one with "unit counter attacks with 200% of original attack strength", order is very important. So the game designer has to add an 'order' value to every object in the game.

So far the favourite library is Fast-JSON-Patch[1]. The method we're trying is to build up a sequence of patches - so every object in the game can add a patch to the queue, and then once the system is ready, it can apply the patches and validate the result.

This particluar library allows the code to directly manipulate the object, and generate a patch from the changes made - so it keeps the code simple and easy to understand.

sumosudo• 4 years ago

I use this as an immutable event store for an append only database implementation.

faichai• 4 years ago

I did the same for a CQRS system. It's been humming away in production for at least 6 years. I've been surprised at how reliable it's been, 0 patch bugs.

Inviz• 4 years ago

How about a library to enable operational transformation for this format? I wrote one once, but got really burned out with some of the stretch goals (log compression for 3+ participants).

sourceless• 4 years ago

We used this to make updates to the NHS Patient Demographic Service (which stores names, addresses, contact information). The records are pretty detailed (FHIR) and there's a lot of business rules and deep nesting.

It worked really well, and our users had no problems making JSON Patch requests.

lootsauce• 4 years ago

prefer doses of reality like this to rants about how the format looks.

dugmartin• 4 years ago

I've used it in the past for document storage updates when documents can be rather large. It is always a toss up, design wise, to patch a single object or to save a log of actions (applied either on the server or the client). The former uses less space and has faster "time to final object state" on the client whereas the latter allows for time travelling and can make debugging much easier.

EERARIX• 4 years ago

Interesting

samatman2• 4 years ago

This could be much more terse:

    {patchset = [ ["+", "/hello", ["world"]], ["!", "/baz", "boo"], ["-", "/foo"] ]}

These are basically reified function calls with known arity, this solution is less XMLish and more lispy, and can be efficiently applied with a bit of destructuring and dispatch on [0].

Spivak2• 4 years ago

This is way less readable to me, you lose all the clarity because now you just have to know all the arguments and order matters.

electroly3• 4 years ago

Hmm, is readability the most important thing here? It seems like terseness may be indicated if this is for PATCH requests. Request bodies are almost never compressed. If the patch is too big, you'd be better off just sending the whole file rather than preparing a patch. The unix diff tool's output is very terse and it's a reliable workhorse.

samatman2• 4 years ago

It's VSO, there is always an operator (six of them), the subject [1] is always a path, and the only one missing an object is remove, which of course has none.

Here:

    const [op, from, to] = patch

Now you can read it again. If you want to hashmap from {+, -, !, |, >, ?} to words, that's understandable.

Edit: We could simplify this to five verbs by letting + take one or two arguments, where the 1adic form sets the value to null. I mean I wouldn't, but we could.

gwbas1c2• 4 years ago

Wouldn't it make much more sense to pretty-print the JSON and ship a normal textual diff?

jannes• 4 years ago

The backend doesn’t necessarily have exactly the same state as the fronted (think about multiple clients).

Textual diffs would only work if the starting point is guaranteed to be the same.

LVB• 4 years ago

Is there a standard pretty print spec?

lichtenberger• 4 years ago

Why would it make more sense? You'd first have to sort the keys, then pretty print, then diff:

https://www.jvt.me/posts/2020/08/24/pretty-print-json-diff/

I'm also using a JSON-diff format in my data store[1], which is accumulated during insertions/updates/deletes in-memory and serialized during a commit, admittedly for large JSON instances (a few Gb and much more).

[1] https://github.com/sirixdb/sirix

gwbas1c2• 4 years ago

You'd first have to sort the keys

A lot of that comes down to internal application architecture. If you're working in Javascript / Typescript, where objects are inherently dictionaries / hashsets with ordering; yes, this is true. You will need to sort before serialization in order for the diff to be useful.

But: If you're working in a strongly-typed language like C# / Java / Rust / C++, and using an object -> JSON -> object serializer; the order of the elements is often quite explicit and never changes.

eadmund2• 4 years ago

It's yet another poor re-implementation of S-expressions and Lisp. This:

    [
      { "op": "replace", "path": "/baz", "value": "boo" },
      { "op": "add", "path": "/hello/bim", "value": ["world"] },
      { "op": "remove", "path": "/foo" }
    ]

is clearly inferior to:

    (patch
      (replace (baz) "boo")
      (add (hello bim) ("world"))
      (remove (foo)))

Although I suppose the verbosity might be helpful in convincing PHBs that one is getting work done, it gets in the way of understanding.

bowsamic3• 4 years ago

How is it clearly inferior? I prefer it to yours

shadowgovt8• 4 years ago

Agreed. The patch key-value syntax is self-describing and extensible. The s-expression syntax's reliance on positional inputs is less self-describing and extension can only be done at the end of the s-expressions.

And the s-expressions don't even solve the problem of terseness for wire-format reasons; protobufs beat them out in that category.

txru• 4 years ago

Generally, any time you have a pairing of an "op" to a variadic list of arguments in a transfer format, you have an s-expression. This method, like many others, introduces a bunch of new characters:

- "op", "path", "value"

- quotes around paths

- colons, commas

- curly braces

In exchange for parentheses.

More generally though, we have an isomorphism. If transformation between the forms is trivial, then use what form is more comfortable for you, but I think being able to identify the 'op' -> 'var-args' pairing is a good thing to keep an eye on. It's macros' bread and butter.

zeven73• 4 years ago

It's inferior because there's a bunch of useless clutter that gets in the way in the JSON patch version.

For the same reason

If a person puts such a question to you as: ‘I have divided ten into two parts, and multiplying one of these by the other the result was twenty-one;’ then you know that one of the parts is thing and the other is ten minus thing.

is inferior to

x(10-x)=21

The succinctness helps your brain focus on the important information so you don't get lost in the noise. If you find it hard to read at first, your brain can adapt to the patterns just fine.

samwillis4• 4 years ago

But now you need an s-expression parser (ok so a very simple thing to do) and a json parser. I don't like JSON Patch but at least its dog-fooding its parser and not invented yet another text representation.

Most of the time the patch documents are not being written by a human anyway.

eadmund2• 4 years ago

But now you need an s-expression parser (ok so a very simple thing to do) and a json parser.

No, the original document should be S-expressions, too:

    ((baz  . "qux")
     (foo . "bar"))

Also, HTML should be S-expressions, CSS should be S-expressions and JavaScript should be Lisp. I have one hammer, and everything looks like a nail

vrfvr• 4 years ago

For me, the former is easier to understand.

Steltek• 4 years ago

The impedance mismatch is that JSON is a fantastic data structure format but a poor code language. Sexp's are a poor data structure format (briefly, lacking a dictionary format and mixing function invocation with lists) but a fantastic code language.

krinchan• 4 years ago

Honest question, why aren't s-expressions in their purest form more prevalent?

This seems to happen constantly on HN:

    Post: "Behold, a data format!"
    Commenter: "This is just poorly implemented s-expressions!"

I know some heavily idolized LISP person out there made the over-quoted statement that all languages end up implementing s-expressions badly.

I did LISP in college. I had my moment of transcendence with it. So I get the fandom for it. But I don't get why people keep paraphrasing this one sentence over and over, yet absolutely no one uses s-expressions outside of LISP. The one person I know who uses LISP in the private sector (via Clojure which many people refuse to recognize as a LISP for some reason?) is being hired to do very academia type research and publish white papers about it. The rest of LISP users are quirky members of academia who complain endlessly about Python's prevalence.

It just kinda seems like if LISP or s-expressions were truly that great, there would be more convergence around it for expressing data structures.

patrickthebold• 4 years ago

This isn't an answer to your question, but I really like this (somewhat old) clojurescript react wrapper: https://github.com/levand/todomvc/blob/gh-pages/architecture...

Notice there is no JSX to define react components, just s-expressions.

(yes I know you can use React without JSX, but I think it's interesting that in a lisp there's not really much need to introduce something like JSX).

Anyway I think it's a neat example on how applicable s-expression are.

bogomipz• 4 years ago

"I know some heavily idolized LISP person out there made the over-quoted statement that all languages end up implementing s-expressions badly."

Not being at all versed in the LISP world myself might you or someone else share where the original quote came from?

draegtun• 4 years ago

> It's yet another poor re-implementation of S-expressions and Lisp.

Or alternatively Rebol, which was one of the inspirations to JSON.

With Rebol you could write these patches like this:

    patch [
        replace /baz: "boo"
        add /hello: ["world"]
        remove /foo
    ]

    patch [
        add /biscuits/1: [name: "Ginger Nut"]
    ]

xvilka• 4 years ago

Reminds me of XSLT.

kitd• 4 years ago

This is the kind of thing gron [1] would be good for. A diff on the gron output before and after should contain all the info needed.

[1] https://github.com/tomnomnom/gron

zzbzq• 4 years ago

JSON Patch is a bizarre Frankenstein's monster made of the cognitive dissonance of REST aficionados.

JSON Patch is not REST. It is not representational state. But it's also not the way any sane person has ever done RPC over HTTP. Typically RPCs accomplish mutation by just supplying whatever is the most ergonomic at the time; if you want to be able to rename an entity, you implement a /rename endpoint., etc.

In Javascript, there was always a normal, obvious way to handle Patch, which was to leverage null vs. undefined properties in javascript. However, in statically typed languages, people wanted to deserialize data into static types that don't support the concept of "undefined." This was the beginning of the end, because deserializing into a static type is actually a bug for an API that has an evolving schema. No longer can you distinguish between a null and undefined property if the a client omitted it.

So instead we end up with yet another preposterous proposed "standard" of Json patch, a completely insane and difficult to read instruction-set-over-json that is truly a hodgepodge.

Remember, mutations like this need to be assembled by the client. That means not only the code but also the user experience. The classic REST-like ways of doing these mutations originate in the "for free" usage of the HTML form element. That's gone, now you need some insane JsonPatcher library, and you need to somehow manipulate your UI around it.

I work in an industry of idiots.

dgb232• 4 years ago

I think your comment is funny, I love a good rant.

I don't think it's as terrible as you describe. But I see your point. We can already do this kind of thing in many different ways that seem preferable over this.

malfist• 4 years ago

Why is it problematic to use a JsonPatcher library over some browser specific feature? This is like refusing to use screws because you own a hammer and don't want to buy a drill.

withinboredom2• 4 years ago

Yep, probably should be using a screwdriver. A drill is probably overkill since it’s designed for drilling and not screwing…

zepolen4• 4 years ago

In Javascript, there was always a normal, obvious way to handle Patch, which was to leverage null vs. undefined properties in javascript

Yeah, no, patch as you describe it is not a patch. Given this:

{field: [1,2,3]}

And given that two clients want to each add 4 and 5 respectively to the document, one will overwrite the other, which breaks the very principle of a patch since you end up with either [1,2,3,4] or [1,2,3,5] instead of [1,2,3,4,5] (or [1,2,3,5,4] depending on who was faster).

I work in an industry of idiots.

Very true.

withinboredom2• 4 years ago

That is a problem with representation and not a real problem. If you have multiple rows in a db, and want to update a row, do you send all rows or describe precisely how to update the rows? No. you send an update and a predicate on which rows to update.

Merad4• 4 years ago

That's why I tend to prefer using PUT for idempotent updates... at least the semantics are (should be?) a little more clear. But if you really want to avoid users stomping on each other's changes you basically have to implement concurrency checks throughout your whole application.

biaachmonkie• 4 years ago

Then you'd have to implement a version or hash field on the object and verify it before updating and fail on mis-match and handle the version mis-match error on the client and re-retrieve and update again.

  GET: {version: 1, field: [1,2,3]}
  PATCH: {version: 1, field: [1,2,3,4]} # succeeds 
  PATCH: {version: 1, field: [1,2,3,5]} # fails with HTTP 409 response
  # Client displays some sort of warning that the update failed, GETs the object again and merges the new state from the server with it's state and can retry the update
  GET: {version: 2, field: [1,2,3,4]}
  PATCH: {version: 2, field: [1,2,3,4,5]} # succeeds

mixedCase• 4 years ago

However, in statically typed languages, people wanted to deserialize data into static types that don't support the concept of "undefined." This was the beginning of the end, because deserializing into a static type is actually a bug for an API that has an evolving schema

No, that's just a bug in your type.

    data FieldPatch a
        = Value a -- A value unmarshable to type `a` was given, apply patch
        | SetToNull -- null was given, apply and patch to null
        | Undefined -- field was not set, do not patch
        | UnexpectedValue RawJSON -- optional, allows manual handling of unexpected types

You can do this in things like Java too with interfaces if you can tolerate openness, or if you want closed types like in the example, through the visitor pattern.

brabel• 4 years ago

You can do this in things like Java too with interfaces if you can tolerate openness

Java supports sealed interfaces now, so this is not true anymore.

eyelidlessness• 4 years ago

JSON Patch is not REST. It is not representational state.

This is a perfectly reasonable take, although it fits well in systems otherwise generally more inclined to use HTTP semantics. I know that’s not REST, but it’s certainly easier to reason about and clearly document (eg OpenAPI) than its RPC analogues.

But it's also not the way any sane person has ever done RPC over HTTP.

Why not?

Aside: I’ve also used it as a format for internal calls and change tracking. It’s surely not ideal, but it was already in the service and trivial to repurpose. Turned out to be an extremely lucky decision, as it also became a crucial tool for forensic analysis in a major recovery effort.

I work in an industry of idiots.

You could just… not be a jerk. You have that option.

yencabulator• 4 years ago

For strictly one way one producer to one consumer state updates, I benchmarked this versus writing the new object to a zstd stream and flushing. Zstd won.

HWR_143• 4 years ago

I've seen several roll-your-own solutions to this, but I have yet to see one that supports "copy" and "move". And rarely do I see one that supports "test". I like those features, even if I think the "op"syntax is clumsy as anything and add/update/remove can just be a JSON object be key:value/new_value/undefined

raydiatian2• 4 years ago

Have seen JSON patch successfully reduce complexity in two separate projects now. My thoughts:

The Good:

1. Depending on your ORM or data access model, JSON patch is a fantastic option. If you’ve got existing services that can hand you deeply nested models, you can start writing PUT/PATCH routes that take JSON patch changesets and let it build things up automatically. Makes it as easy as checking a resource out of storage, applyPatch or applyOperation, check it back in.

2. You don’t really need to worry about bad types getting dumped into your base object, as you should be deferring type safety to the route DTO level anyway, and everything can trickle from there.

3. The ‘-‘ semantic works fine for signaling an intent to insert a new element.

4. 99% of the time the “protocol” seems to just work.

5. YMMV, but there’s a good chance that switching to JSON patch will let you delete front end code as well, since the front end models can in theory also use the very commands they pass to the back end to locally update models. It integrates very well with a redux pattern and seems like a base technology for undo/redo.

The Bad:

1. On the one hand ‘/‘ as the primary delimiter irks me. I suppose ‘.’ characters in a property name are more common than slashes, but still not my favorite. Furthermore, the escaped characters not using backslashes is weird. You write ~0 if you need to refer to ~ literally and ~1 for /, if I am not mistaken.

2. There’s a missing contextual mode for the ‘add’ op: creating missing objects along the path you’re talking about. For instance if your base object is {}, then add at path ‘/foo/bar’ will fail since foo is undefined. Should be a mode where I can say “yes, please create missing stuff along the way, thank you.”

“On the JSON patch, feel your GraphQL cravings subside in as little as thirty minutes!”

raydiatian2• 4 years ago

One final “bad”: the update operation (known as replace) is also slightly lacking in my opinion. I’d like to be able to update an object with a partial fragment, eg:

The op

{ “op”: “replace”, “path”: “/foo”, value { “a”: 3 } }

Applied to a base object

{ “foo”: { “a”: 0, “c”: 1 } }

Will update “a”, but will throw away existing property /foo/c.

This is dumb. The other problems like it are dumb. Like so dumb it’s hard to not ruminate, so everybody here that complains is right. But not much so that the technology is void of use as some seem to suggest.

It’s so funny that JSON Patch 2 hasn’t come out with more extensions/improvements. Perhaps this Ycomb thread is the little push over the edge that we need to get a ball rolling!

muti• 4 years ago

Furthermore, the '/' delimiter seems an odd choice given JSON is JavaScript Object Notation. Why not use JavaScript syntax for pointers?

gegtik• 4 years ago

An alternative approach is to use jsonpath; for python, the jsonpath-ng implementation is useful but could use more friendly documentation.

it contains XPath-style selectors and the ability to directly modify matched nodes in a Dict, including filtering (as indirect deletion) and upserts; the only thing that it's missing is sufficient addressability with regards to Lists (arrays), so I see myself building a similar 'op' processing but using jsonpath instead of jsonpatch language.

add one to the pile i guess

_Chief• 4 years ago

In case you're trying to open the RFC links on the site, the IETF site no longer supports http and does not redirect to https, hence you see a 404 error. You can manually open https versions of the urls to visit them

JSON Patch - https://datatracker.ietf.org/doc/html/rfc6902 JSON Pointer - https://datatracker.ietf.org/doc/html/rfc6901

Tried PR'ing to fix site, but repo didn't seem active

nikolay• 4 years ago

Isn't it amazing how JSON reinvents pretty much every technology XML ever had? This includes the well-forgotten XQuery, which is in a way what jq and JMESPath offer, too.

btbuildem• 4 years ago

I've used this in the past on projects -- overall a helpful set of functionality, and simplified some front-end stuff (less code written is always good in my books).

What Is JSON Patch?

Related Stories