We diff and patch source code all the time, but doing the same thing for JSON data isn't as straightforward. More often than seeing the differences between JSON texts, we're interested in seeing (and sharing and reapplying) the differences between the objects encoded in the text. We want to gloss over differences in whitespace and order of object members and see the real difference in the data. Unix patch and diff operate only on text, and best on text of consistent and short line length, and aren't very useful for parsing out differences between objects encoded in JSON. If diff and patch aren't going to work, what will?
JSON Patch is a format (identified by the media type "application/ json-patch+json") for expressing a sequence of operations to apply to a target JSON document; it is suitable for use with the HTTP PATCH method.
This format is also potentially useful in other cases in which it is necessary to make partial updates to a JSON document or to a data structure that has similar constraints (i.e., they can be serialized as an object or an array using the JSON grammar).
RFC 6902 comes from developers interested in REST-style interaction with JSON representations of resources but might be more generally useful. How does it work? If you're familiar with XPath and XPointer, you might see some similarities. A JSON patch doc contains an array of path, value, and operation tuples. The path tells you how to traverse into the JSON document and the operations specify what is done to objects using the values.
For an example, I'm going to take GeoJSON from Lyzi Diamond's Learn GeoJSON project and make diffs using Stefan Kögl's python-json-patch package. RFC 6902 has implementations in other languages, too, such as jsonpatch-js (npm).
import json import jsonpatch import os.path import requests import subprocess # original if not os.path.exists('hackspots-44de76ef53.json'): r = requests.get('https://github.com/lyzidiamond/learn-geojson/raw/44de76ef53b20bdaf51e0cde4aa634df210cd9d4/geojson/hackspots.geojson') data = r.content with open('hackspots-44de76ef53.json', 'wb') as f: f.write(data) # next commit if not os.path.exists('hackspots-e9514f5c31.json'): r = requests.get('https://github.com/lyzidiamond/learn-geojson/raw/e9514f5c317ee980b94ed6580950cfd9fbde53db/geojson/hackspots.geojson') data = r.content with open('hackspots-e9514f5c31.json', 'wb') as f: f.write(data) diff_text = subprocess.check_output([ 'jsondiff', 'hackspots-44de76ef53.json', 'hackspots-e9514f5c31.json']) diff = json.loads(diff_text) print(json.dumps(diff, indent=2))
The result looks, to me, like a very reasonable representation of an operation that adds one feature to a GeoJSON feature collection.
A hypothetical change of this same feature's coordinates from e9514f5c31 to https://gist.github.com/sgillies/7797688#file-foo-geojson looks like this:
There's a potential for RFC 6902 style GeoJSON diffs to become very large if highly detailed lines and polygons are being modified (via simplication or reprojection). But GeoJSON itself is quite verbose already and a verbose patch representation feels not surprising. If you already think GeoJSON sucks, JSON patch for GeoJSON is going to look exta sucky because of the lengths of paths to individual coordinate items. I don't think there's any way around that, although GeoJSON's recursive coordinates member helps a little – consider that the path to the first 'x' coordinate of the exterior ring of the first part of a multi-polygon is "just" '/features/42/geometry/coordinates/0/0/0/0'.
I'm not sure how RFC 6902 patches will play out at scale, but I think it's worth further consideration.