I think that by “between python and java” they meant “faster than python and slower than java”. I think Java still beats JS unless you get lucky.

You’re totally right that benchmarking and profiling is hard even for native code. I think this post fetishizes whether or not a piece of code got JITed a little too much. Maybe the author had a bad time with microbenchmarks. There’s this anti pattern in the JS world to extract a small code sample into a loop and see how fast it goes - something that C perf hackers usually know not to do. That tactic proves especially misleading in a JIT since JITs salivate at the sight of loops.

> I think that by “between python and java” they meant “faster than python and slower than java”. I think Java still beats JS unless you get lucky.

Yeah, that is what they meant, but it is a little misleading. Javascript's performance is usually within a single digit multiple of java's, whereas python is often significantly slower. Javascript is somewhere between Java and an abacus too.

That still doesn't make sense to me.. how can JIT in general be slower than Java when Java is JITed?

The peer comment about static typing is correct, but there's more, err, flavor to be enjoyed in the challenges of JITing JS. Here's some deopt scenarios to keep you up at night.

To whet your appetite, what happens if you redefine Math.round? Java prohibits this for obvious reasons, but a JS joker may write:

    Math.round = () => Infinity;

JS engines really do inline Math.round, and must be prepared for this nonsense.

It gets worse. Maybe you check for a property which is usually not found:

    if (!val.uuid)
       val.uuid = uuidgen();

Hidden classes can almost reduce this to a pointer comparison. But what if someone adds a uuid property on Object.prototype? Every check is busted! v8 handles this with "validity cells", and it's ugly, requiring that every object know about every object which "inherits" from it.

Now if you are a monster you may choose to write:

    Object.defineProperty(Array.prototype, "42", {value: "lol"});
    console.log([][42]); // yes it prints "lol"

Every array gets a default value for 42. Think about how you would JIT numeric code in such an environment...

Python suffers the same issue. There's some discussion about dealing with it in this PEP and the pages it links: https://www.python.org/dev/peps/pep-0509/

Python does not seem to be even trying to be competitive in performance though so it's less of an issue for them.

They do, it's just not a priority. The PEP I linked (as well as most of the work of its author, Victor Stinner, in the last years) are motivated by JITing.

There's also the "frame evaluation API" PEP [1], whose purpose is to allow pluggable evaluators in CPython without forking the entire interpreter, like Unladen Swallow had to.

[1]: https://www.python.org/dev/peps/pep-0523/

Python is orders of magnitude slower than JS, though. It has the same problems, but doesn't even try solving most. PyPy would be a better VM to compare against.

Java deals with almost exactly the same issue. If you define an interface and a single implementing class, the code will be compiled to always call that classes method, and the deoptimized / recompiled if you load another class that implements that interface. JS could deal with these issues in a similar manner.

You can redefine Math.round. It's an ordinary method from rt.jar. You can change rt.jar or you can redefine it with java agent.

Not as straightforward as with JS, but Java JIT still have to account for those things.

May be it will assume some things about standard classes for optimization, but for user classes that will be true anyway.

You can really redefine an existing Java method at runtime? Well TIL!

The Java JIT has static type information to work with, the JS JIT can only infer type information via heroic efforts. Static types do mean something, especially when working with primitives and other unboxed data types.

This. Javascript engines have to do stuff like https://mathiasbynens.be/notes/shapes-ics to assume many objects will have the same key names and types and create an optimized class layout, with some kind of expensive fallback if anyone ever randomly stores a key or value that violates the assumption. They can't even be sure you'll always be doing integer math (unless you resort to bit masking a la asm.js). Hinting (like __slots__ in Python) might have helped some.

This was one of the main motivations for writing dart. The V8 implementers were tired of having to write these kinds of hacks. Even though dart was initially a dynamically typed language, the shape of the objects was stable.

> The Java JIT has static type information to work with

I don't know that the java JIT has type feedback. However regardless Java being statically typed means Java code is structurally closer to what a JIT wants and can analyse, it's much more difficult (and thus rarer) to fuck around with types and objects generated on the fly at runtime for instance, you're not going to add new attributes to an instance whereas that's just tuesday in javascript.

Java JITs definitely do have type feedback. They profile the types of objects at virtual call sites and do guarded inlining, as well as removing expensive interface casts, etc. The JVM doesn't really have a distinction between classes generated on the fly and those that were on disk, as they all go through a classloader anyway. JVMs, for the most part, blow their brains out at the end of the day and start with zero the next VM run.

Well, you're right that JS JITs can only infer type information via heroic efforts, but AFAIK the Java compiler throws away any type information from the source code, which means that the JVM JIT needs to inver type information again from the byte code.

Still, the JVM JIT is faster than JS due to reasons explained in the sibling comment of the parent one.

Java is only throwing generic type information away in the declaration of classes and methods. The code that is accessing generic types is compiled with casts to the expected type. List<String>.get(1) will generate a cast to String therefore nothing is actually missing in the generated code. It is only missing when you use reflection to e.g. deserialize a List via Jackson by passing List<String>.class. That unfortunately won't work because the generic type parameter is not part of the declaration, only the generated code.

> but AFAIK the Java compiler throws away any type information from the source code

You're thinking about generics. .class files preserve a whole bunch of type information (I'm building a .class decompiler in my free time, and I'm looking at that very same data in my debugger ATM).

You probably know this, but it's not obvious to the random reader.

Even generic types are available in the Java .class file and are accessible from the reflection API. Spring for example uses this quite heavily.

It depends on where you are at.

Type information is present in fields and class inheritance

For example, a class like this

`class Foo implements Bar<String>`

Retains the fact that the generic type is a String.

That information is completely lost at method invocation. So a method that takes a `Bar<String>` ultimately compiles to a method that takes a `Bar` and knows nothing of the String.

To get that generic information down you have to engage in some fun tricks using either the class or field method I mentioned earlier. (Usually you do this with a second type parameter where it matters).

Java JITs don’t throw away type information. The bytecode carries strong types, though you have to do some work to get them out (some abstract interpretation). But that’s something you can do statically; no need for profiling or speculation.

In JS you can only get the types by profiling.

Generics are erased in Java. Not so in the CLR.

Yeah but the thread is about types, not generic types specifically.

Also it’s not really true that genetics are erased. There’s that horrible thing javac does so that the VM can support reflection for generics. I get your point though.

Do you mean the passing of classes as additional arguments? I haven't seen the horrible thing.

Overall I think C# got this one right. Generics are right there in the binaries.

bUt ActuALLy, I think the rightest right thing is to do specialization up to representation at link time (or compile time), when the whole program is available, ala MLton. Virgil does this. Of course this is not possible in a dynamic code loading environment but I only have so many fucks to give in this life.

No, I meant that the Class format has metadata about what the genetics were so some java.lang.reflect thing can query about what the genetic type was. I don’t think it comes with soundness guarantees.

I agree C# got this right.

I agree that doing specialization up to link time is ideal from a certain standpoint.

All generics in Java are of type Object from the start, you can't call any methods on them not implemented by Object. So it is wrong to say they are erased, they were never there to start with.

You can add a constraint to a generic declaration, which is an upper (or lower) bound on the allowable type arguments.

e.g.

  class A<Y extends X> {
    // in this scope, Y is known to be of at least type X,
    // so, we can call methods on expressions of type Y
    // that belong to type X (and not Object)
    Y m() { ... }
  }

Type arguments are omitted from usage sites. The technique is literally called erasure in papers and documentation.

  a = new A<Foo>();
  f = a.m(); // should return a Foo, in bytecode returns Y
             // and compiler inserts a cast from Y -> Foo

Generic code is slower in Java because of these extra casts. To get back (most of) the performance, the JVM has to inline enough methods to be able to track the types from start to finish. It can't always.

And that's why Java is slower than it could be. If I'm using String[] array, it can't contain anything but String objects, so JVM does not have to check type everytime I'm accessing that array. That's not true for ArrayList<String>, where compiler must check returned object type from `get(index)` (because it really is Object and can contain anything), but it could be true with better language design.

This is not true, bytecode stores fields and classes as fully qualified strings: https://stackoverflow.com/a/17406592

Yeah what you said.

You're not wrong, but I always wonder how the verbosity and crazy class hierarchy of Java code doesn't make things slower.

I know that compilers are smart, but the API doesn't make things easier at all

The Java standard library isn't particularly verbose or insane. It's the "enterprise java" world that deals with the sort of excesses you're thinking about.

Verbosity makes the programmer slower not the program.

It depends. For example, you can verbosely write a loop to copy data from one array to another, or you can use memcpy. The later is typically implemented as hand optimised assembly - it's a pretty small number of operations in x86_64. The former - maybe the compiler optimizes, maybe it doesn't. If it does optimize it, it's definitely more complexity in the compiler and slows it down.

In general having some higher level, well optimized helpers can certainly reduce verbosity and increase speed. That said, some types of verbosity just make the programmer write what the compiler would translate to anyway. Or can end up being unnecessary - e.g. a strongly typed language with no inference certain slows the programmer with no effect on the program (though maybe an effect on the type checker).

A fast memcpy used to be complicated to implement but these days I heard a simple

    rep movsb

will do the trick. This assumes that the source register points to source, destination register points to destination and counter register is set to number of bytes to copy.

I wish that was true but my data says it isn't:

- A verbose SIMD copy loop is often faster even on modern Intel CPUs that have the more modern rep implementation.

- A simple byte copy loop will beat both SIMD and rep when you're copying smaller amounts at a time.

Yes I understand some verbosity is structural and won't change anything, for example

SomeClass x = new SomeClass();

won't make anything slower (ignoring auto for now)

Now, I read Java code and I can't help but wonder how things are harder

For example, setters and getters, what would be a simple memory write becomes a function call. Not complaining when you actually need it.

Reading the examples here (and those are not too bad) https://developer.android.com/reference/java/net/HttpURLConn... it seems you have to actually fight the API to get anything done

Why do you need to cast the return of a url.openConnection() to a HttpURLConnection? (I mean, how many connection types exist?)

So for getters and setters a Jit can inline the function call and in the end you have a direct memory access. In your example you have to cast because URLConnection could be a HttpURLConnection or JarURLConnection. But again a good Jit would speculate that it is always a HttpURLConnection and deoptimize if not.

The JVM also has full information about the class hierarchy at runtime. If you add a class that overrides a function it will deoptimize that overriden function to a virtual function but otherwise the JVM will just treat it as a static function and inline it if necessary.

In modern Java I don't see why you would directly use an HttpURLConnection. The stdlib provides an easy to use HttpClient.

https://docs.oracle.com/en/java/javase/11/docs/api/java.net....

> I mean, how many connection types exist?

Two? https://developer.android.com/reference/java/net/URLConnecti...

The design of JavaScript is not very friendly to high performance. Even though Java is itself not easy to generate efficient code from, producing efficient code from JavaScript is far more difficult.

The claim wasn’t about JIT in general, but rather about JS.

Then it’s problematic to be so JS and V8 centric. Also the post specifically talked about JS being faster than Python and slower than Java so that’s what this thread is about.

I’m not advocating for the post or anything else, I was just clarifying a misunderstanding.

That was not a general statement about Javascript performance. The entire article is about the unpredictability of the JIT. When the JIT hits a bad codepath then it really does perform like python or when it hits a good code path (most of the time) then it performs like java. This unpredictability is what is causing the issues, not the general performance.

Pypy JIT is way faster than CPython

It is, but that’s not saying a lot and it’s still far far slower than Go/Java/C#, and it also isn’t compatible with lots of existing Python code. Such as anything that would like to talk to a Postgres database. Pypy is great, but it can’t fix a broken ecosystem and general lack of leadership in the Python community. :(

(Broken ecosystem = lots of things don’t work with lots of other things; package managers are another example; Mypy another)

What's the problem with Postgres on PyPy? Running fine in production here.

There are no supported drivers for it as far as I could find. The common answer is to use psycopg2cffi, but that hasn’t had a commit in like 4 years. The alternatives I could find were in the same shape or worse.

There's a few pure python drivers. Check https://wiki.postgresql.org/wiki/Python and https://wiki.python.org/moin/PostgreSQL

E.g. pg8000 looks like it's being maintained?

I recall looking at pg8000 in particular, but it didn't inspire confidence for our production use case. It had only 20 commits in 2019 and fewer than 100 stars. I also didn't have a good understanding of the performance tradeoffs (pypy/pg8000 vs C library).

Poetry and mypy are fine. Postgres works on pypy. But generally.. yeah you're right.

My experience with Mypy has not been pleasant. It doesn't work with a whole host of packages including zip packages (PEP 441) or those that aren't built for Mypy directly (you have to sprinkle empty .pytyped files in various places in your project IIRC or mypy will ignore the type hints you provide; I ran into this with the `black` package. Further, getting it to find installed packages is tremendously difficult--it generally just fails with "couldn't find package" or something similarly useless and the docs it links you to are not helpful in my experience. Mypy also barfs on recursive types, so you can't express things like JSON, nor can you constrain generic types based on protocols.

Similarly, Postgres doesn't work for Pypy unless you're willing to use libraries that are unmaintained or only very lightly maintained, and even then who knows what the performance tradeoffs will be. Those are pretty substantial caveats for a production use case.

I haven't used Poetry, but I've heard good things. I'm very cautious though, because I've also heard high praise about pip/virtualenv, pip-tools, pipenv, etc and every single one had spectacular problems on the happy path (30 minutes to resolve dependencies with pipenv is my favorite).

> who knows what the performance tradeoffs will be

Sure that's true, but performance is about measurement.

Poetry has some missing pieces still (I can list them if you're interested but I'm on my phone) but it's the first time I've felt ok recommending a packaging tool to beginners. The experience is way better than the alternatives like pipenv.

> Sure that's true, but performance is about measurement.

I have a very low degree of confidence that a pure Python solution (even JIT optimized) is going to compete with a native solution, not to mention that by all appearances it isn't sufficiently well-supported to meet our criteria. Measuring the performance for me feels like a waste of time; it's easier just to forego Pypy until the ecosystem matures.

> Poetry has some missing pieces still (I can list them if you're interested but I'm on my phone) but it's the first time I've felt ok recommending a packaging tool to beginners. The experience is way better than the alternatives like pipenv.

This is good to know. I'll have to give it a try.

>JS with JIT is waaay faster than Python. Not “between python and java” as purported.

You are correct that JS is not between Python and Java. Python is faster than Java which is faster than JS. Though some people seem to think calling APIs written in C is "not Python" but if the ecosystem provides the library and I call it from Python then it's Python enough for me!

By that logic, Python, Javascript, and Java are all the same speed, just call out to C, or assembly perhaps if you really want to tune some code.

No, not at all. Java FFI is extremely expensive.

Yeah. And so is JS’s. I think CPython’s design makes it a champ at native interop and a loser at pure compute perf. That’s a valid trade off for lots of important situations.

Are you referring to V8's/Node's C FFI? Why is that extremely expensive?

It's not a trade-off. There's no reason why you can't have a blazing fast and expressive language that also has good C FFI. A single good programmer could make a language that was as expressive as Python (that is, not very), faster, and with a good C FFI in under 6 months.

You couldn’t be more wrong. Java and JS have expensive FFI because of the specific costs of synchronizing the VM entry/exit with the GC and JITs.

Aren't you saying it's possible to make a perfect language with no tradeoff, and all that in just 6 months?

No I'm saying it's easy to make a language better than Python in six months.

Then where is it?

What you're implying is that the "speed" of a language is purely defined by the speed of the C FFI? This view is absolutely ridiculous and I don't think you would find a single other programmer on this green earth that would agree with you.

I think everyone agrees that when they want to do some data analysis they pip install numpy and get to work. It's only the navel gazers who are concerned about cpythons performance instead of treating python as a 'race to C' wherever performance is concerned.

Get used to this approach since it crops up everywhere. If you target a gpu, you try to keep the work in the gpu. When we move to use bpf and io_uring things will be a race to the next kernel program. If you target the CPU then your code is a race to C or a race to the function that calls the relevant cpu instruction that will make your code fly.