Note to responders--HN won't let me respond to you, because "I'm posting too fast".
That’s not to say that CPython isn’t slower than a compiled language but in common usage the gaps usually end up being much smaller because most of us aren’t paid to run fib() and things like I/O dominate or code ends up being in the same C libraries (libjpeg, OpenSSL, etc.).
I’ve seen multiple cases where Python beat C/Go because the app used something like dictionaries which was heavily optimized C code in the standard library.
Again, compiled languages should usually win but the details matter, especially since most people don’t do the exact same work you do. Simply tossing around big numbers with no support contributes noise rather than insight to the conversation.
I had a csv file with ~16k entries. Each entry has a lat, lon, name and id.
The task: given a (lat, lon) pair, list all entries within 5km.
The simplest straight forward solution is to iterate the list, measure the distance of each item to the given point, and add it to the return list if the distance is within the desired range.
No crazy optimization or anything.
The results were something roughly like this:
The D version runs in 2~4ms when compiled with optimizations turned on to the max, and about 6~8ms without optimizations (just default compiler settings).
The Go version runs in about 6~8ms. (The Go compiler has no option to produce optimized code).
The python version runs in about 50ms.
That's 10x slower than Go and 20x slower than D.
This is python 3. Using pypy shows absolutely no difference in performance.
Obviously the time to load the data from the csv file into a list is not counted in the above results.
> I’ve seen multiple cases where Python beat C/Go because the app used something like dictionaries which was heavily optimized C code in the standard library.
That's exactly backwards.
The Go and D versions use structs which are compact data structures that pack the data together. So the entire array fits in one contiguous block of memory.
Python has no concept of structs, so you have to deal with dictionaries.
In general, writing optimized code is not a good idea. You should almost always write the most plain and straight forward code.
Because fast languages are fast by default, there's no optimization required. The straight forward solution is fast enough.
If you are writing Python, your code will be slow by default. You will need to think about ways to optimize it if you want better performance.
The go version runs in about 3ms, not 6ms.
The D version runs in 2ms.
So overall the python version is 10x slower than native, not 20x slower.
Which is still kind of bad.
The only thing I measured is iterating the list and matching entries within range of the input coordinate.
from math import radians, cos, sin, asin, sqrt
def distance(lat1: float, lon1: float, lat2: float, lon2: float) -> float:
# taken from: https://stackoverflow.com/a/4913653/35364
# convert decimal degrees to radians
lon1 = radians(lon1)
lon2 = radians(lon2)
lat1 = radians(lat1)
lat2 = radians(lat2)
# haversine formula
dlon = lon2 - lon1
dlat = lat2 - lat1
a = sin(dlat/2)**2 + cos(lat1) * cos(lat2) * sin(dlon/2)**2
c = 2 * asin(sqrt(a))
r = 6371008 # Average radius of earth in meters
return c * rThe Ruby GIS libaries already have C implementations for haversine but I just wanted to show how easy it is to hook out to a C function for math heavy stuff.
The javascript version runs in 15ms.
> In the real world, things like haversine are implemented as C extensions
Having to write C-extensions is, well a point of friction, and adds complexity.
If you just used a native language to begin with, the simple straight forward code would work just fine.
Btw the source is here, and it also includes the csv file. https://bitbucket.org/hasenj/geodb/src
C extensions are a core part of Ruby. A big chunk of the std lib is implemented in C. You shouldn't be afraid of writing a tiny bit of C. Ruby and Python are scripting languages. They're literally designed for scripting another language.
The JIT for Ruby 3 is well underway, so in a bit it'll be a complete non-issue anyway.
> Simply tossing around big numbers with no support contributes noise rather than insight to the conversation.
I resent that you characterize my comment as "flamebait" for lacking supporting evidence for my claim, when I was merely responding to your unsupported claim that Python competes with compiled languages. I chose to be broad because (as you point out), the range varies considerably by application and the margin for error is high, so it would be misleading to be artificially precise. It's perfectly reasonable that you want supporting evidence, but it doesn't make my post "lower quality" than yours.
As for "what I benchmarked", my comment was based on a decade of experience with these languages, including translating many proprietary systems from one language to another (sometimes a faster->slower language and sometimes a slower->faster language depending on whether I was optimizing for performance or security or maintainability).
> That’s not to say that CPython isn’t slower than a compiled language but in common usage the gaps usually end up being much smaller because most of us aren’t paid to run fib() and things like I/O dominate or code ends up being in the same C libraries (libjpeg, OpenSSL, etc.).
> I’ve seen multiple cases where Python beat C/Go because the app used something like dictionaries which was heavily optimized C code in the standard library.
I don't dispute that there are cases where Python beats naive C/Go. In particular, Python's csv lib is optimized C code, and it beats Go's CSV parser handily. If all you're doing is parsing CSVs and not actually calling into Python, you should probably stick with Python. While it's true that many applications are I/O bound, my figure was based on CPU time--it wouldn't make sense to judge languages by applications that are inherently I/O bound (although Go's async-by-default is much nicer than Python's async API). Besides, many applications are not I/O bound.
My point was simply that while it's easy to sit around talking about microbenchmarks that's not helpful for most developers. I've seen this cycle repeat so many times over the last couple decades because we're culturally prone to focusing on very small things and ignoring the larger picture:
Most developers are not employed to make a single thing as fast as possible but instead have to ship new features on a regular basis, handle a wide range of inputs correctly, etc.
Most of the performance problems people notice are due to application logic rather than low-level language characteristics, and that tends to mean things like I/O (API calls, poor file access patterns, etc.) or inefficient memory access patterns rather than how fast a loop can be.
I've seen slow applications in compiled languages replaced with a dynamic language and a big performance win not because of the underlying language characteristics but because writing in a lower-level (C, C++, Go) or more convoluted (Java J2EE) environment took enough longer that the development team was struggling just to complete features and the {Perl,PHP,Python,JS} team had considerably more time to optimize the core design.
I've also seen developers in all of those create huge labyrinths which made it hard to reason about the code's performance characteristics, and so my main concern these days is far less caring about the language than the overall amount of complexity you're asking developers to reason about and the related questions about the project and staffing.
Given unlimited time and budget, yes, a compiled language will probably win but for most projects the better question is “what tools will allow me to meet my goals with acceptable performance?” and that's a very different question.
To be clear, I never mentioned or implied microbenchmarks.
> Most developers are not employed to make a single thing as fast as possible but instead have to ship new features on a regular basis, handle a wide range of inputs correctly, etc.
I absolutely agree that performance is just factor among many, but the scope of this entire post is "performance". Most of your points seem to pertain to "the biggest performance problem I see is with poor engineering practices". This is definitely a problem, and you're right to prioritize addressing it over choosing a fast language. No language is going to fix stupid, although I will say that Go makes it less appealing to write stupid code, thanks to its restrictive type system. So once you've fixed your stupidity problem, the language-level problems will start to crop up. :)
And by the way, since you mentioned memory access patterns and I/O, I want to point out that these are also language-level concerns. Since everything in Python is a reference into a random location of heap, memory access performance is terrible. Go lets you specify whether something is a pointer or a value, which means you can specify which things are collocated. This is important for good cache performance. Regarding I/O, Python makes you choose between async for performance or sync and your process does no work until the I/O completes. Go lets you write normal synchronous code, and the runtime/scheduler make async calls under the hood so code which doesn't depend on the result of your I/O can execute while other code blocks.
> Given unlimited time and budget, yes, a compiled language will probably win but for most projects the better question is “what tools will allow me to meet my goals with acceptable performance?” and that's a very different question.
It is a very different question, but as new compiled languages improve on ergonomics without compromising performance and as new interpreted languages improve on performance without compromising ergonomics, the answer will increasingly become "not CPython" (Python is the fastest growing language in 2017, but that's largely due to a surge in data science popularity; however, since Python is popular with the data science community largely because of its libraries and not language features, other languages will catch up [and indeed are catching up] quickly).
> As for "what I benchmarked", my comment was based on a decade of experience with these languages…
No doubt others have based their comments on their experience, which they trust more.
How do you know - "that's largely due to"?
It depends, in some benchmarks they are 10 times slower.
Should we say that "it depends, in some cases, Java is infinitely faster than C/C++/Rust"? Not unless we want to deliberately obscure the issue (or we want to warn someone about strange results). Benchmarks are about finding representative results across a variety of tasks.
If you think there are good benchmarks that show these languages are 10x slower in realistic cases, then you should share them, rather than just asserting that they're out there.
> And you can construct cases where the JVM is infinitely faster than C/C++/Rust, because the JIT optimizes through a virtual call and detects dead code.
How would that be a "representative result across a variety of tasks" according to your own definition of a benchmark? What do you mean by "across a variety of tasks"? That sounds like taking the average of some metric for different benchmarks, I'm not sure that is a good idea. I think it makes more sense to benchmark your specific use case, which is what I meant by "it depends".
> If you think there are good benchmarks that show these languages are 10x slower in realistic cases, then you should share them, rather than just asserting that they're out there.
I never said anything about "realistic" and "good", but they are really not unthinkably hard to find: http://benchmarksgame.alioth.debian.org/u64q/binarytrees.htm...
http://benchmarksgame.alioth.debian.org/u64q/compare.php?lan...
If you have to write a large project from scratch in Python it will be slow, but Python has gone pretty far in the direction where for many large tasks 90% of the functionality will be obtainable through standard libraries. Doing the remaining 10% in basic Python often keeps it competitive at runtime with compiled languages but produce much simpler, shorter and easier to maintain code.