News for hackers. Stuff that matters.
Re:PyPy's New JSON Parser by umvi on Tuesday October 08, 2019 @10:38AM (#21192791)
> In the last year or two I have worked on and off on making PyPy's JSON faster, particularly when parsing large JSON files

I think this is the key takeaway. If you are parsing huge JSON documents, this will make a difference.

Otherwise, it's super hard to beat Python3's native implementation. I've tried them all - hyperjson, orjson, ujson... none of them provide any significant gains on parse speed of smallish JSON documents, even if you are parsing thousands of small(ish) JSON documents in a row.

Re:PyPy's New JSON Parser by tachyonbeam on Tuesday October 08, 2019 @12:37PM (#21194105)
Something else to note here is that the Python interpreter is slow compared to native code, so if you're parsing a lot of small JSON files, whatever difference in speed the JSON parser makes is masked somewhat by the interpreter overhead (assuming you're doing something with that JSON you just parsed). When you're using a JIT like PyPy, the overhead of the rest of your code becomes less, and so then the bottleneck could become something like JSON parsing, I/O, etc.
Re:PyPy's New JSON Parser by TkTech on Tuesday October 08, 2019 @05:47PM (#21197648)
This is the main bottleneck in pysimdjson[1]. 95-99%[2] of the time is spent creating Python objects rather than parsing the documents.

There was no trick I could find (although treating all strings as bytes instead of using PyUnicode_DecodeUTF8/unicode_decode_utf8 was a fairly large gain on string-heavy documents) to reduce the cost of bringing the document into Python land, so I added a quick hack to push the filtering down into C, since most of my use cases only need a small part of the full document.

[1]: https://github.com/TkTech/pysimdjson [2]: https://github.com/TkTech/pysimdjson/issues/22

Re:PyPy's New JSON Parser by glangdale on Tuesday October 08, 2019 @06:18PM (#21197902)
(one of the original authors of simdjson here)

I thought a bit about this - I think you could do a lot faster if you had a 'bulk' method to create lots of Python objects in one hit. I don't know it that's feasible, as I don't know much about the innards of Python, but there's no reason one couldn't keep goofing around with SIMD code and branch-free stuff all the way until you've created objects.

This may be ferociously unmaintainable (i.e. depend on things that Python implementations don't advertise as a part of the API) but I don't know. I doubt you're the only one to have this problem.

Re:PyPy's New JSON Parser by TkTech on Tuesday October 08, 2019 @07:27PM (#21198404)
Like you said, it would unfortunately be unmaintainable, requiring a fork of CPython. Aggressive optimizations like this could go into pypy, but IMO CPython is still trying to be a "reasonably readable" reference implementation.

I think the time is currently best spent trying to push more of the work back into native code, extending the simple filtering that's in there now to also handle mutation (similar to jq), reducing the amount of back-and-forth with the interpreter.

Re:PyPy's New JSON Parser by ufo on Tuesday October 08, 2019 @06:21PM (#21197926)
This whole discussion makes me thing that perhaps there would be a point in parsing JSON into a "JSON database" object instead of a tree of individual hashmaps. In theory it would allow for a more memory efficient representation and more faster search or transformation operations.
Re:PyPy's New JSON Parser by weberc2 on Tuesday October 08, 2019 @02:44PM (#21195633)
> so if you're parsing a lot of small JSON files, whatever difference in speed the JSON parser makes is masked somewhat by the interpreter overhead

Does this vary based on the size/quantity of the input JSON files? I would expect this to hold regardless of size/quantity of the input files? Further, isn't CPython's implementation already native code? If so, I would expect the bottleneck to be allocating the output objects or pressure on the GC or something?

Re:PyPy's New JSON Parser by AaronFriel on Tuesday October 08, 2019 @03:15PM (#21195990)
Of course, the smaller the JSON file, the shorter the amount of time spent in native code, and proportionally more time will be spent in the interpreter around whatever looping construct is used to feed data into json.loads (and presumably process that data).

    # interpreter time spent here looping
    for file_name in stuff:
        # interpreter: do stuff with file name, load file into string, etc.
        str_data = ...
        # interpreter: dispatch call to native function
        # native: actually perform json.loads
        data = json.loads(str_data)
        # interpreter: time spent processing the data
        process_data(data)

    # what happens as native time goes to zero (json parse time decreases)?
    # answer: interpreter time dominates
Re:PyPy's New JSON Parser by weberc2 on Tuesday October 08, 2019 @05:12PM (#21197333)
So "interpreter time" is defined as "time not spent parsing JSON" and thus "the less time you spend parsing JSON the less time you spend parsing JSON"? Of course that's correct, but it's so obvious as to be unsubstantial. Surely the OP is making a more interesting point...
Re:PyPy's New JSON Parser by AaronFriel on Tuesday October 08, 2019 @09:12PM (#21199024)
No, that's it - for most json documents, the time the interpreter spends in all the stuff surrounding (and executing) json.loads dominates the time spent in native code parsing. That's why replacing json.loads with a faster native implementation doesn't result in noticeable improvements - it's speeding up the fastest bit, and if it's already 0.1% of the wall clock time, then you can't get much more than about 0.1% faster.

PyPy is a JITed language though, so the interpreted bits are much faster. OP is making the point that Python 3/CPython is so slow that speeding up json.loads is a premature (and unnecessary) optimization.

Re:PyPy's New JSON Parser by VHRanger on Tuesday October 08, 2019 @01:08PM (#21194447)
I would think that's because the overhead tends to dominate when calling on small JSON, whether it's function call overhead or python's interpreter, etc.
Re:PyPy's New JSON Parser by IanCal on Tuesday October 08, 2019 @11:18AM (#21193211)
Really? I just tried with a sample of one of my files, using python 3.6. With text in memory, 1M smallish strings decoding is 12s with ujson and 25s with the built in json. orjson and rapidjson seemed about the same for this test.
Re:PyPy's New JSON Parser by umvi on Tuesday October 08, 2019 @11:32AM (#21193387)
Hmm, I'm starting to think my definition of "small" is different from everybody else's. I consider "smallish" JSON to be at most 100KB (and on average 1-10 KB). 1 MB I consider to be "large"... basically anything that doesn't parse near instantaneously I consider "large".
Re:PyPy's New JSON Parser by Uehreka on Tuesday October 08, 2019 @12:20PM (#21193916)
I think GP might've been saying 1M small JSON documents, rather than a single document containing 1M strings.
Re:PyPy's New JSON Parser by nicoburns on Tuesday October 08, 2019 @01:58PM (#21195058)
If you use a language like Rust (which is really easy for these kind of jobs), then 1mb will parse almost instantly.

I've been working with a 90gb JSON file recently. I'd probably start counting a file as large at around 100mb.

Re:PyPy's New JSON Parser by aisengard on Tuesday October 08, 2019 @02:15PM (#21195258)
I have to ask - why? I understand if you're dealing with an external resource that you can't change, but I'd still like to ask them the same question. JSON's strength, it seems to me, is that it's great for human consumption. It's a very readable data structure when serialized! But who is ever going to read 90GB of JSON?
Re:PyPy's New JSON Parser by nicoburns on Tuesday October 08, 2019 @05:44PM (#21197634)
Not out of choice! It was a backup of a firebase database (also not my choice). I totally agree that JSON is not a good format for files this large, but it was the only format the data was available in so what can you do.
Re:PyPy's New JSON Parser by nly on Tuesday October 08, 2019 @02:33PM (#21195473)
Not OP, but I deal with a variety of proprietary binary/encoded files in the 10s of MB to the 10s of GB range at work, and we have started decoding to JSON as the first step in our normalization, indexing and analytics pipeline (To be honest we essentially just throw everything in to MongoDB). Daily we're in the terabyte range.

It's the perfect solution because you can use the same decoder + something like jq for debugging and manipulation on the command line.

Re:PyPy's New JSON Parser by e12e on Tuesday October 08, 2019 @03:38PM (#21196287)
A question - do know of any good resources for jq? I haven't quite been "angry enough" to use jq much beyond using it for formatting json - but that's also partly because I can't seem to quite grasp the syntax.

It isn't quite "awk-ish", not quite cut/join/sort/uniq. But also not quite graphql? Which I suppose would be the "natural" language for a json-query tool?

I feel like I'm missing a piece - and missing out...

Re:PyPy's New JSON Parser by chucky_z on Wednesday October 09, 2019 @12:43AM (#21200098)
The manual is pretty dang good, tbh.

https://stedolan.github.io/jq/manual/

Re:PyPy's New JSON Parser by e12e on Wednesday October 09, 2019 @02:26AM (#21200604)
I agree that it is clear, but I still struggle to figure out (by myself) how to do (in an awk sense) simple things like:

Filter out empty or null values:

https://stackoverflow.com/questions/39500608/remove-all-null...

I seem to either end up with unwieldy, long/verbose solutions - or shorter ones that does not work :)

Re:PyPy's New JSON Parser by chucky_z on Thursday October 10, 2019 @10:48PM (#21221111)
Oh wow, that's terrible!

I tend to get 90% of the way with jq and then awk/sed/grep my way to the desired result (jq -r).

Re:PyPy's New JSON Parser by orf on Tuesday October 08, 2019 @02:40PM (#21195567)
It's not necessarily that it's great for human consumption, more that it's super interchangeable and ridiculously easy to get up and running, and for the most part 90Gb of space is so cheap that it's not worth the overhead of using a binary format.

I'm guessing that OP wasn't parsing a single 90Gb object, but a stream of many newline delimited JSON objects.

Plus the human element helps.

Re:PyPy's New JSON Parser by nicoburns on Tuesday October 08, 2019 @05:43PM (#21197627)
> I'm guessing that OP wasn't parsing a single 90Gb object, but a stream of many newline delimited JSON objects.

I wish that were true. Alas it really was a 90gb object. Actually, the first thing I did was split it by top-level key before processing further.

Re:PyPy's New JSON Parser by weberc2 on Tuesday October 08, 2019 @02:50PM (#21195697)
Especially considering that JSON compresses well.
Re:PyPy's New JSON Parser by ken on Tuesday October 08, 2019 @01:23PM (#21194632)
Isn't that tautological? Clearly, no JSON parser provides a significant gain on parse speed of small JSON documents, when you define large as those where parse time is significant.
Re:PyPy's New JSON Parser by coldtea on Tuesday October 08, 2019 @03:55PM (#21196476)
>Isn't that tautological?

No, it's no.

>Clearly, no JSON parser provides a significant gain on parse speed of small JSON documents, when you define large as those where parse time is significant.

A rather bizarre deduction.

E.g. "define large as those where parse time is significant".

So let's say a large file is one whose parse time is > 10 seconds.

So, while looking at small, ie. < 10 second parse time files, if a JSON parser takes files that others load in 500ms and load them in 20ms, then that's "a significant gain on parse speed of small JSON documents"...

We didn't have to change any of the definitions...

Re:PyPy's New JSON Parser by umvi on Tuesday October 08, 2019 @01:56PM (#21195038)
Well, Python2 -> Python3 introduced a dramatic performance increase, even for small JSON documents.
Re:PyPy's New JSON Parser by IanCal on Tuesday October 08, 2019 @03:37PM (#21196271)
These were small though, I just made a small sample of a million of them and still saw a 2x improvement.
Re:PyPy's New JSON Parser by IanCal on Tuesday October 08, 2019 @03:36PM (#21196259)
Oh I think you misunderstood, they are all small, there's just a million of them. I think they were all about 1k.
Re:PyPy's New JSON Parser by acdha on Tuesday October 08, 2019 @11:35AM (#21193414)
Note the qualifiers about document size — anything measured in millions probably isn't “smallish”, and unless your app does nothing other than decode those strings the total runtime change is unlikely to be anywhere near 2:1.
Re:PyPy's New JSON Parser by IanCal on Tuesday October 08, 2019 @03:45PM (#21196354)
Each document was small, there was just a lot of them.

They can be huge changes. With slightly larger docs we had a setup that had to return batches of results back. The bottleneck was absolutely the single thread part of the API encoding and decoding JSON.

Even if the docs were just the size of my example that's 25s for 1m docs. With a few threads I can easily run 10k+/s of the useful part so that'd be 100s + 25s + encode time. With a few more threads hitting 50k/s that's 20s useful time + 25s + more.

With larger docs it becomes far more of a problem. More importantly the fix is a few seconds to implement.

Re:PyPy's New JSON Parser by acdha on Tuesday October 08, 2019 @04:46PM (#21197042)
Again, note the qualifiers: nobody is arguing that faster JSON parsing isn't a good thing, only that anything measured in the millions is probably not considered “small”. A significant amount of code is never profiled since it's fast enough and if someone does find the stdlib json library to be using enough time to care about it is certainly easily replaced.
Re:PyPy's New JSON Parser by IanCal on Tuesday October 08, 2019 @04:57PM (#21197182)
The small qualifier seemed to be on the size of the documents, and on small documents I see a 2x or often higher improvement. At a scale of 10k docs that's still 100ms difference, and in my case can be a huge proportion of the total time spent (more with larger docs, and more again if you include encoding time).
Re:PyPy's New JSON Parser by naniwaduni on Tuesday October 08, 2019 @08:26PM (#21198732)
Plenty of things measured in millions are small. A million bytes is a small amount of memory. There are plenty of domains where a working definition of "small" is roughly "can be slurped into memory on a single node without jumping through hoops for provisioning".

A million iterations that total to order of ten seconds seems quite reasonable for a quick performance comparison. It's just underspecified for the rest of us to take away much beyond that.

Re:PyPy's New JSON Parser by bjoli on Tuesday October 08, 2019 @11:18AM (#21193207)
I have had pretty large speedups in the past when switching JSON implementations. About 5x. It doesn't matter much when the times are 0.2 Vs 1s, but when you are working with a couple of MB of JSON the difference will definitely be noticeable.

Python's STDlib is nice, but rarely spectacular. An old naive CSV parser I wrote for Chicken scheme (using a simple state machine) beat python's C-based one.

Re:PyPy's New JSON Parser by acdha on Tuesday October 08, 2019 @11:34AM (#21193408)
> Python's STDlib is nice, but rarely spectacular. An old naive CSV parser I wrote for Chicken scheme (using a simple state machine) beat python's C-based one.

The stdlib is also conservative: I've run into quite a few cases where someone had reported huge gains until they realized they weren't properly handling escaping, Unicode, etc. and once updated the gap narrowed considerably.

Re:PyPy's New JSON Parser by ehsankia on Tuesday October 08, 2019 @11:30AM (#21193368)
You're basically agreeing with the above, and I have experienced the same. The only place where this makes a difference is when the JSON file is in MBs. For smaller file, it's not really worth the extra library.
Re:PyPy's New JSON Parser by IanCal on Tuesday October 08, 2019 @05:09PM (#21197291)
Depends on how many of the small docs you deal with, I see useful differences on small documents because I process reasonable amounts of them. A 10s change on a million docs for just decoding for a single library is easily worth it for me.
Re:PyPy's New JSON Parser by mhd on Tuesday October 08, 2019 @10:42AM (#21192829)
From the Perl side, there's been an interesting talk by Graham TerMarsch[1] where he tried quite a few of the allegedly fastest JSON parsers and compared them to the standard solution (JSON::XS). Apparently a lot got lost "in translation", when converting the C/C++ data.

[1]: https://www.youtube.com/watch?v=BToZ_E3vU0Y

Re:PyPy's New JSON Parser by rurban on Wednesday October 09, 2019 @02:57AM (#21200756)
Interesting comparison of most fast json parsers. Just https://github.com/lemire/simdjson is missing. Purple beating RapidJSON.

The standard is btw. my Cpanel::JSON::XS and he didn't compare with cperl which would be much faster and more secure. And he misinterpreted why glibc made a big difference. - march=native got better, not malloc.

Re:PyPy's New JSON Parser by fnord123 on Wednesday October 09, 2019 @10:05AM (#21203365)
I would like a json-game like csv-game[0] that benchmarks all of these together. Here is an example with a /lot/ of C++ parsers:

https://github.com/miloyip/nativejson-benchmark

[0] https://github.com/miloyip/nativejson-benchmark

Re:PyPy's New JSON Parser by xtreak29 on Tuesday October 08, 2019 @10:43AM (#21192849)
Python 3.9 would also include some optimizations for parsing large strings : https://bugs.python.org/issue37587
Re:PyPy's New JSON Parser by jfkw on Tuesday October 08, 2019 @11:04AM (#21193049)
while reading that bug, I am reminded how amazed I am that the Roundup issue tracker [1] has continued to serve Python development after all this time and through transitions in hosting and workflow. Roundup is neat, and I'm glad it exists.

[1] http://roundup.sourceforge.net/

Re:PyPy's New JSON Parser by linsomniac on Tuesday October 08, 2019 @11:15AM (#21193162)
Ka-Ping Yee, the original author of Roundup, has done some pretty awesome things. I remember one time I was BSing with a recruiter in the hallway at PyCon, and Ping walked by behind the recruiter. The recruiter stopped talking to me mid-sentence and gave chase: "Ping! Ping! Can I talk to you for a minute ..."
Re:PyPy's New JSON Parser by xtreak29 on Tuesday October 08, 2019 @11:32AM (#21193386)
There is an accepted PEP to transition to GitHub issues : https://www.python.org/dev/peps/pep-0581/ . But there is another PEP to keep using roundup to address concerns raised.
Re:PyPy's New JSON Parser by Nicksil on Tuesday October 08, 2019 @12:46PM (#21194205)
>But there is another PEP to keep using roundup to address concerns raised.

Seems to be this one: www.python.org/dev/peps/pep-0595/

Please correct if I'm wrong.

Re:PyPy's New JSON Parser by xtreak29 on Tuesday October 08, 2019 @01:45PM (#21194910)
Yes, maintainer of the bpo also showed some demos of REST API addition to build other features and gsoc student working on few features like markdown support during this year core dev sprints. There is a separate pep for actual migration https://www.python.org/dev/peps/pep-0588/ . I am not sure when it would take effect or so given the depth of bpo in core dev workflow. There is also an open issue to track workflows that are missing in GitHub but present in bpo : https://github.com/python/core-workflow/issues/359
Re:PyPy's New JSON Parser by sontek on Tuesday October 08, 2019 @04:20PM (#21196740)
Python RapidJSON: https://github.com/python-rapidjson/python-rapidjson

Is pretty amazing. Its the first library we add to most of our projects. I wonder if it would make sense to port it to using cffi so that pypy can benefit?

Re:PyPy's New JSON Parser by slovenlyrobot on Tuesday October 08, 2019 @09:56AM (#21192333)
This post touches briefly on ujson. I found it a long time ago, and while I can't vouch for the quality of its implementation, it's very popular and seems to work just fine. Any time I've encountered a bottleneck due to parsing, dropping ujson.loads() in has made it completely vanish from profiling output.
Re:PyPy's New JSON Parser by ambivalence on Tuesday October 08, 2019 @10:21AM (#21192590)
Please note that according to the benchmark in this blogpost, CPython's builtin JSON decoder is faster than ujson.
Re:PyPy's New JSON Parser by gshulegaard on Tuesday October 08, 2019 @11:28AM (#21193336)
That result is actually why I am healthily skeptical of the performance comparisons specifically in this post. Like the previous comment I too have been in a situation where JSON encode/decode on large payloads were significant enough that we decided we needed to figure out something more optimal. Our solution was to use ujson instead (of the built in) and that was good enough that we moved on.

When I say skeptical what I mean is that I have no doubt the measurements are accurate (as in not fabricated); however, I wonder what the full context of the benchmarks are and whether or not it's representative of situations like mine where the built-in parser started to fall over so we replaced it. It may also be the case that the PyPy parser would also beat ujson in this case, but if we are limiting our test case to just those where the CPython implementation is marginally faster than ujson then I'm not sure it's that useful a benchmark.

I too cannot comment on the quality of the implementation or the relative performance against other parsers...but it solved a performance problem for at least one project of mine in the past.

Re:PyPy's New JSON Parser by kmod on Tuesday October 08, 2019 @02:06PM (#21195139)
Agreed, I've gotten huge speedups from ujson in the past and the numbers in the post are markedly different from my experience, making me think that they are measuring something different / there is something significant about this use case that is different than the ones I've had in the past.
Re:PyPy's New JSON Parser by weberc2 on Tuesday October 08, 2019 @02:54PM (#21195738)
The post is profiling large payloads? Maybe ujson is optimized for smaller payloads?
Re:PyPy's New JSON Parser by wyldfire on Tuesday October 08, 2019 @10:35AM (#21192763)
It's not strictly the case (`reddit` case).

But the difference is pretty insignificant compared to Node and PyPyFull.

Re:PyPy's New JSON Parser by superwayne on Tuesday October 08, 2019 @10:05AM (#21192420)
I think there are even faster JSON parsers out there nowadays. I have recently used orjson and found it to be extremely fast for our use case (serializing a dict with about 100 MB).
Re:PyPy's New JSON Parser by gsaga on Tuesday October 08, 2019 @04:13PM (#21196658)
>This is completely transparent to the Python programmer, the dictionary will look completely normal to the Python program but its internal representation is different.

Shouldn't it be opaque.

Re:PyPy's New JSON Parser by ptx on Tuesday October 08, 2019 @05:36PM (#21197552)
The optimization is transparent because the representation is opaque, I guess?