News for hackers. Stuff that matters.
Re:Rust is now overall faster than C in benchmarks by vitus on Sunday January 03, 2021 @03:31PM (#25624946)
I recall an anecdote about how Haskell actually outperformed C on various tree benchmarks because it was using a better implementation. At some point, the C programmers got fed up with the airs of superiority from Haskell programmers, ported the Haskell implementation, and reclaimed their position.

I wouldn't be surprised if there's something similar happening here.

Re:Rust is now overall faster than C in benchmarks by spiffytech on Sunday January 03, 2021 @03:43PM (#25625050)
That kind of tit-for-tat in benchmarks seems like it's counter to the goal of benchmarks: what kind of performance could I expect to see using technology $FOO? Crucially, that question depends on how someone will realistically implement $FOO.

I like PyPy as an example: on the surface, implementing a Python runtime in Python and expecting performance gains seems crazy. PyPy manages to outperform CPython because although a C implementation should theoretically be faster, realistically the increased expressiveness of Python lets the PyPy devs opt into optimizations the CPython devs find out of reach.

I don't know C or Rust well enough to comment on these specific scenarios, but if two technologies can be fast, and one makes that speed accessible while the other encourages implementors to leave performance on the table, that's much more useful information to me than seeing a back-and-forth where folks compete to implement optimizations I'll never make in my own projects.

Re:Rust is now overall faster than C in benchmarks by moonchild on Sunday January 03, 2021 @04:17PM (#25625332)
> PyPy manages to outperform CPython because although a C implementation should theoretically be faster, realistically the increased expressiveness of Python lets the PyPy devs opt into optimizations the CPython devs find out of reach.

Pypy outperforms cpython for the simple reason that pypy is a jit where cpython is a basic bytecode interpreter. Anything else is icing on the cake.

The only reason python seems slow as an implementation language is because it was traditionally slow; the reason it was traditionally slow is that cpython, the only major implementation, is slow. Common lisp, for instance, is similarly dynamic (moreso, in fact), also is also generally natively compiled by itself, and is quite performant.

Re:Rust is now overall faster than C in benchmarks by aidenn0 on Monday January 04, 2021 @11:52AM (#25633959)
Common Lisp (hereafter Lisp for concision) is differently dynamic, not strictly more dynamic.

One quick example: standard-class objects in Lisp are mostly more dynamic than python objects[1] but structure-class objects are quite static, and compilers will use this to their advantage.

On top of this, functions in python are in fact objects; in python you can do this just fine:

  def foo():
    foo.bar = 3

  def bar():
    print(foo.bar)
There are obviously other ways to accomplish this in lisp, but the fact that someone might do something like this can inhibit optimizations.

If you read the Lisp specification you will see dozens of places where they allow for opting out of dynamism in ways that help the compiler optimize. Type declarations are one and the fact that a function defined in a single file is considered to have a fixed definition for the scope of the file is another.

1: It's not quite as easy to add new slots to Lisp standard-class instances on the fly as it is to python class instances, but it's doable, and Lisp, of course, supports class redefinition in a way that few other languages (including Python) do.

Re:Rust is now overall faster than C in benchmarks by spiffytech on Sunday January 03, 2021 @07:13PM (#25626583)
PyPy has a JIT because using Python as the implementation language made it simple to build a runtime generator instead of a single runtime implementation, so RPython (the underlying PyPy tech) can generate a JIT and non-JIT runtime from the same source code. Conversely, the CPython team doesn't have the resources to make a JIT runtime the bespoke way that's more typical for these projects.

https://morepypy.blogspot.com/2011/04/tutorial-part-2-adding...

Re:Rust is now overall faster than C in benchmarks by naniwaduni on Sunday January 03, 2021 @10:17PM (#25627614)
CPython isn't a JIT because its goals as a portable reference implementation would be much harder to achieve with a JIT.

To put it another way, there's a reason PyPy trails CPython language features on the scale of literal years. This is not explained by the affordances of the language they're written in! They're just projects with very different requirements which, on some platforms, happen to run a lot of similar-looking code.

Re:Rust is now overall faster than C in benchmarks by ghostwriter on Monday January 04, 2021 @08:32AM (#25631057)
> there's a reason PyPy trails CPython language features on the scale of literal years.

the reason is under-funding, and the lack of developers to the cause as a consequence, not the portability complexities of jit.

Re:Rust is now overall faster than C in benchmarks by vitus on Sunday January 03, 2021 @03:50PM (#25625109)
You raise a good point. I've always been fascinating by PyPy's performance, personally -- anecdotally, I've achieved ~10x speedups from just running a script with `pypy` instead of `python`. I always attributed that to better performance of the JIT, but I could be wrong.

I have nothing against Rust personally, but it's ultimately not an apples-to-apples comparison if they're not implementing the same algorithm, or even if they're not using the same mechanisms (e.g. Rust explicitly using SSE intrinsics, which are certainly available to C in just as idiomatic a fashion).

Re:Rust is now overall faster than C in benchmarks by gameswithgo on Sunday January 03, 2021 @03:58PM (#25625187)
comparing apples to apples is pointless, they are both apples.

and you can always compare “equivalent” algorithms as some languages may not be able to efficiently express the same algorithm as another. i know what you are after, but trying to have some benchmark that is “fair” according to some spec that is important to your needs will just be seen as pointless to others. benchmark game at least lets us see what the performance ceiling is of various languages, and anytime you object to a language having a worse implementation you may go fix it.

i think what anyone with a clue understands is that C C++ and Rust all have roughly equivalent performance ceilings. Nobody really thinks Rust is faster than C now.

Re:Rust is now overall faster than C in benchmarks by vitus on Sunday January 03, 2021 @04:18PM (#25625347)
> Nobody really thinks Rust is faster than C now.

The submission title would beg to differ.

(I know, it explicitly calls out "benchmarks" as the context.)

I think languages like Rust or Swift have significant advantages around safety over C/C++, while not sacrificing much in terms of performance. But if one language's benchmark contributors are willing to put in more effort than another's to eke out additional performance, then you're going to see skewed results in favor of whichever has the more fervent evangelists or whichever language has more to prove.

If the goal is to compare performance of two languages which can express the same optimization in exactly the same way, and only one uses it, then the benchmarks fail in that respect.

Re:Rust is now overall faster than C in benchmarks by gameswithgo on Sunday January 03, 2021 @05:52PM (#25626023)
go fix the c implementation then.

no single person can maintain optimally implemented solutions over dozens of languages. its up to we the people to help out. you want perfectly equivalent comparisons, make it happen

Re:Rust is now overall faster than C in benchmarks by platinumrad on Sunday January 03, 2021 @07:38PM (#25626744)
I don't want perfectly equivalent comparisons. I want people to stop citing this set of imperfect ones as if they mean anything.
Re:Rust is now overall faster than C in benchmarks by wang_li on Sunday January 03, 2021 @07:11PM (#25626577)
Yeah. At a fundamental level these programs are running on the same hardware. Unless you’ve an ultra simplistic language (basic or logo), one could implement identical algorithms and methodologies and techniques and get the same performance.
Re:Rust is now overall faster than C in benchmarks by sli on Sunday January 03, 2021 @04:15PM (#25625314)
I admit I'm destroying the metaphor here but there are a lot of useful comparisons to be made between different cultivars of apple and those comparisons have implications on their applications.
Re:Rust is now overall faster than C in benchmarks by zeroimpl on Sunday January 03, 2021 @05:14PM (#25625736)
That's the whole point of the metaphor - comparing apples to apples is good, apples to oranges is bad.
Re:Rust is now overall faster than C in benchmarks by jcelerier on Monday January 04, 2021 @06:21AM (#25629941)
I'm comparing apples to oranges every time I'm trying to decide which fruit to buy at the supermarket though.
Re:Rust is now overall faster than C in benchmarks by gameswithgo on Sunday January 03, 2021 @05:51PM (#25626010)
and its a bad metaphor. there are many reasonable comparisons to be made between any two things even when both are not fruits
Re:Rust is now overall faster than C in benchmarks by ghostwriter on Monday January 04, 2021 @08:41AM (#25631160)
Technically speaking, there are limits and you cannot compare incomparable. You can find distinctions between two phenomena, a.k.a. inherent traits that make them different-by-definition from one another, but you cannot compare (liken, collate, contrast) them in any meaningful way. Like, what's the outcome of comparing "soft" with "warm"? And hence, comparing apples to apples are not pointless, quite contrary - they are comparable items and it makes total sense to compare them in the context of superiority by desirable value axes.
Re:Rust is now overall faster than C in benchmarks by igouy on Monday January 04, 2021 @01:52PM (#25636098)
> Like, what's the outcome of comparing "soft" with "warm"?

You can have "soft" or you can have "warm" — which do you want?

You can have an apple or you can have an orange — which do you want?

Re:Rust is now overall faster than C in benchmarks by mkl on Sunday January 03, 2021 @05:56PM (#25626056)
> That kind of tit-for-tat in benchmarks seems like it's counter to the goal of benchmarks

Yep. There are good reasons why this set of benchmarks is called The Computer Language Benchmarks Game.

Re:Rust is now overall faster than C in benchmarks by igouy on Monday January 04, 2021 @03:33PM (#25637459)
Re:Rust is now overall faster than C in benchmarks by FartyMcFarter on Sunday January 03, 2021 @03:53PM (#25625139)
> I like PyPy as an example: on the surface, implementing a Python runtime in Python and expecting performance gains seems crazy.

Not if it includes a JIT compiler, as PyPy does. I don't know much about PyPy, but if most of the time is spent in JITted machine code, the fact that it's written in Python may not affect performance much.

Re:Rust is now overall faster than C in benchmarks by pjmlp on Sunday January 03, 2021 @05:25PM (#25625793)
Kind of, PyPy is a metacircular JIT compiler, it uses RPython a Python subset.
Re:Rust is now overall faster than C in benchmarks by cygx on Sunday January 03, 2021 @05:50PM (#25626003)
realistically the increased expressiveness of Python lets the PyPy devs opt into optimizations the CPython devs find out of reach

Not really: It's the compilation strategy (that whole meta-tracing JIT compiler thing, compared to a simple bytecode interpreter) that makes the difference, not the surface syntax of the implementation language - which is actually not 'real' Python, but a restricted, less dynamic subset known as RPython.

Also note that the CPython is deliberately kept 'dumb'.

Re:Rust is now overall faster than C in benchmarks by harporoeder on Sunday January 03, 2021 @03:43PM (#25625042)
There was a long period where a fairly unknown theorem proving language ATS (1) was beating C in many test cases on the benchmark game (2) the benchmarks were removed though (3). I expect many languages could be made to win with sufficient effort.

1. http://www.ats-lang.org/ 2. http://web.archive.org/web/20121218042116/http://shootout.al... 3. https://stackoverflow.com/questions/26958969/why-was-the-ats...

Re:Rust is now overall faster than C in benchmarks by steinuil on Sunday January 03, 2021 @03:55PM (#25625161)
ATS is not really a theorem proving language, it's almost a superset of C with a very long list of type system and language features that lets you write anything from C code with no safety to very high level recursive functional code with lifetime tracking which will translate to efficient C code, if the transformations are proved to be correct. It's a weird beast, but I'm not surprised it outperformed some C implementations, because it is basically C with a lot more features.
Re:Rust is now overall faster than C in benchmarks by ghostwriter on Monday January 04, 2021 @08:54AM (#25631328)
> ATS is not really a theorem proving language

What is "real theorem-proving language" in this context? ATS has ATS/LF that is designed for writing formal proofs in the language.

http://ats-lang.sourceforge.net/DOCUMENT/INT2PROGINATS/HTML/...

Re:Rust is now overall faster than C in benchmarks by FartyMcFarter on Sunday January 03, 2021 @04:48PM (#25625573)
> I wouldn't be surprised if there's something similar happening here.

In this case it seems like benchmark code is allowed to use intrinsics, which can degenerate into a situation where a benchmark in language X is more "glorified x86 Assembly code" than actual code in language X.

This is not very useful for comparing languages IMO. Especially since all of Rust, C, C++ can use this strategy and become almost identical in both code and performance.

Re:Rust is now overall faster than C in benchmarks by gameswithgo on Sunday January 03, 2021 @05:57PM (#25626069)
So three languages with identical performance ceilings end up with essentially identical performance results. sounds like a job well done.

if you want to know something more than that, like how performance tends to end up after inputs of identical effort and talent you will need to recruit a lot of people and money to run really large scale experiments and when you are done people on HN will just endlessly find nits to pick with your results whenever they don’t agree with their preconceived notions

Re:Rust is now overall faster than C in benchmarks by johnisgood on Monday January 04, 2021 @08:35AM (#25631092)
It is always like that. There is no way that X is faster than C. It most definitely has to do with algorithm. What I can believe is Fortran being faster than C in some circumstances, but that is about it. Can you guys please stop this nonsense? No, your equivalent Rust code is not faster, sorry.
Re:Rust is now overall faster than C in benchmarks by Blikkentrekker on Sunday January 03, 2021 @04:43PM (#25625539)
The Haskell code on some pathological examples of these implementations that I have seen has so many unsafe construct, strictness annotations, inlining annotations and so forth that it's practically C in a different syntax.

It is not idiomatic Haskell at all and loses all of the touted benefits.

Is there a separate benchmark that only accepts idiomatic code?

Re:Rust is now overall faster than C in benchmarks by igouy on Monday January 04, 2021 @02:01PM (#25636214)
Is there a GHC compiler switch that ensures only "idiomatic code" is compiled?

One can, with sufficient effort, essentially write C code in Haskell using various unsafe primitives. We would argue that this is not true to the spirit and goals of Haskell, and we have attempted in this paper to remain within the space of “reasonably idiomatic” Haskell. However, we have made abundant use of strictness annotations, explicit strictness, and unboxed vectors. We have, more controversially perhaps, used unsafe array subscripting in places. Are our choices reasonable?

http://www.cs.ru.nl/P.Achten/IFL2013/symposium_proceedings_I...

Re:Rust is now overall faster than C in benchmarks by Blikkentrekker on Monday January 04, 2021 @07:13PM (#25640173)
It shows the performance of that variant of Haskell.

But when using that variant all the correctness guarantees that Haskell is known for are no longer on the table.

Re:Rust is now overall faster than C in benchmarks by pjmlp on Sunday January 03, 2021 @05:30PM (#25625837)
Just like C is pimped up BCPL, a language designed to bootstrap the CPL compiler, 10 years younger than ESPOL/NEWP, 10 years younger than Jovial, 6 years younger than PL/I, among other possible examples from the 60's.

Also C was known for not being a fast language on 80's home hardware, ruled by Assembly.

So low level coding for OS isn't coding like C and is just the way for mechanical sympathy, regardless of the language.

Re:Rust is now overall faster than C in benchmarks by loxias on Sunday January 03, 2021 @04:43PM (#25625537)
It's exactly the same thing. I've been seeing this tossed around for years now, and one of these days it'll make me grumpy enough to fix the benchmark.
Re:Rust is now overall faster than C in benchmarks by _wldu on Sunday January 03, 2021 @03:39PM (#25625011)
Yes, it seems everyone is always trying to beat C, but really, no one can.
Re:Rust is now overall faster than C in benchmarks by pjmlp on Sunday January 03, 2021 @05:34PM (#25625866)
Well, junior Assembly developers had it pretty easy on 80's hardware.

Also C++ and Fortran don't have to spend much effort to achieve that.

Re:Rust is now overall faster than C in benchmarks by lawn on Sunday January 03, 2021 @03:42PM (#25625039)
To be fair, if there's a language that has a fair chance it's Rust.
Re:Rust is now overall faster than C in benchmarks by madacol on Sunday January 03, 2021 @07:34PM (#25626719)
Looked at the git repo https://salsa.debian.org/benchmarksgame-team/benchmarksgame and it seems there's only one committer. So if all scripts are written by the same guy, then his biases are all over the place, and it needs more code diversity before we can arrive to any conclusion.

Am I missing something?, is there a list somewhere of the people who have contributed to this?

Re:Rust is now overall faster than C in benchmarks by Narishma on Monday January 04, 2021 @06:41AM (#25630093)
Click on any of the benchmark programs and it will have the contributors' names at the beginning.
Re:Rust is now overall faster than C in benchmarks by madacol on Monday January 04, 2021 @07:36PM (#25640402)
Thanks!