I wonder what optimizations Zig does that lets it generate machine code / LLVM bitcode faster than both C++ and Rust (which certainly have larger teams backing them), at least in the case of this Gameboy emulator project.
[1] half the time I’d make the compiler crash; the other half it would generate a binary which crashes at runtime, with weird heisenbug behaviour like “adding a print statement to log how far down a function I am causes the code to stop crashing at all” — like right now there is a load-bearing print statement which shows the address of the SDL Window object, because otherwise the compiler seems to optimise the Window out of existence and then a few lines later it segfaults on the null pointer…
Is the most glorious thing I ever heard.
that sounds like you have undefined behavior in your program
https://stefan-marr.de/2022/10/cost-of-safety-in-java/
Unsafety buys you a little more performance, but the baseline should the safe version, not the unsafe version. It is like having to explain on a case by case basis why you aren't using lead pipes for this application.
Always default to safety. The difference between safe and unsafe native code is usually single digit percentage points. Or weeks on a Moore scale.
Heck, I see 1.33x differences in run times between the first & second run of the rust branch of his benchmark, but only 1.05x diffs in run times between 1st & 2nd Nim branch.
In my experience, reasoning about things like this is rarely as simple as "bounds checks" which are (often) the most highly predictable branches.
PHP is traditionally used solely for websites. Some of those have grown rather large, to the point that having engineers optimize the language is cheaper than buying more servers.
Python, on the other hand, is first and foremost a scripting language. When performance does matter, you often end up using a wrapper around a C library, like NumPy. This means there is relatively little money in optimizing the Python interpreter.
It would seem sensible that since Facebook poured a lot of resources in optimizing PHP, Google would have done the same for Python.
Also, Python is the first or second most popular language.
Well they sort of half assed tried years ago - they employed GvR at one point. Then they gave up and they continue to lean heavily on C++, Java and there is thing they developed in the interim: Go.
> Also, Python is the first or second most popular language
Hogwash. For all the open source and other development that occurs and is “indexed” on internet discussion forums there is countless boring ass shit behind the scenes in sweatshops around the world and corporate back rooms. PHP, Java, and C# are still probably more popular to start.
Much of which is also in Python.
I say this as someone that has made their career mostly on the “top 5” TIOBE languages - but that ranking is a bunch of crap for the overall commercial software world.
C? Give me a fucking break. Never in 20 years have I known there to be more C than Java jobs.
And now everyone and their dog including doctors and other non tech-savvy professionals are doing Intro to Python courses (and then never touching it again) so they too can feel like they know something about ML or stats. Lot of noise.
According to 2022 Stack Overflow developer survey, if we exclude HTML, SQL and Bash, top five languages (as in being used by most developers) are: Javascript, Python, Typescript, Java and C#.
Checking here, if we go by pushes on GitHub, JS, Python, Ruby, Java, PHP are the top for 2021.
This is of course biased to only stuff on GitHu band public (assuming, but doubt they're publishing private data on BigQuery).
No idea how much of that is pet projects vs production of course.
By what metrics? Redmond quarterly from June shows Python at #2, and PHP at #4.
https://redmonk.com/sogrady/2022/10/20/language-rankings-6-2...
> By what metrics?
Hmm, why don’t you answer your own question?
> Redmond quarterly from June shows Python at #2,
That clearly is one that puts it “first or second”, yes.
But still wanted to know by what metrics the GP was making their claim.
I've looked at various sources since the last 5 years and not much changed. I hoped some languages like Nim, Crystal or Zig would pick up a bit of steam, but no.
Go and Rust moved up a bit and now seem safer bets to invest some time into.
For Python, the C integration probably makes things harder.
It's generally faster than Python in doing the same things as Python does, unrelated to web too.
(Measures Python 3.10.4 against PHP 8.1.5 - so expecting these to change a bit).
Meta/FB is also pretty invested in Hack (was once a php dialect, again, out of the loop), maybe they contributed a thing or two?
TIL MySQL 6
At Wikimedia we adopted it which has cut our CPU usage by half and has saved a few hundred of servers. We had some Facebook engineers helping which involved patching the Linux kernel while at it. Those were good times.
Eventually PHP 7 followed up with a similar approach and had more or less the same performance as HHVM. Facebook went then to focus on the Hack language (a dialect of PHP with strong typing) and eventually phased out back compat with Zend.
From what I remember, Sara Golemon at Facebook has done a lot of outreaching to Open Source project and gave us a lot of assistance (as well as others at Facebook).
I remember it being slower than PHP4 for a while
Of course, people then go and build up sculptures of objects that will be thrown away after every request, and that stuff makes everything slow (that style of code is why PHP wasn't good enough for Facebook, IMHO), but you can build trash sculpture in any language.
I recall a couple of times over the past 15-20 years where a current python significantly outperformed a current php version, but my recollection is that php was usually a bit faster, or sometimes a lot faster.
PHP 7 was released on dec 2015, and gained significant speed bumps and better memory usage, with the average php execution time being cut in half, or more, generally without any code change whatsoever. It was quite remarkable.
The path from 7.x-8.1 so far has generally seen incremental speed bumps again - usually somewhere between 3-8% improvements per release. Obviously this is going to be dependent on use cases, but overall it's been a fairly steady set of speed improvements over the last 7 years.
In PHP, doing `$a->b` accesses field `b` of object `$a`. The implementation is essentially type check (making sure `$a` is an object) and hash table lookup. If this fails, then it calls `__get` method if it exists.
In Python on the other hand, `a.b` involves `__getattr__`, `__getattribute__`, `__mro__`, `__get__`, `__set__`, `__delete__` and `__dict__`. Here is a description of how the attribute access works: https://docs.python.org/3/howto/descriptor.html#overview-of-....
$ brew install pypy3 pypy3: The x86_64 architecture is required for this software.
$ pyenv install pypy3.9-7.3.9
I like using PyEnv for managing my Python versions. It will natively compile Python builds and should be doing the same on M1 (which is what I'm using). `pyenv install --list` shows you what is available.
EDIT: Not sure why they don't have newer versions of PyPy there (I don't use PyPy) but all it takes is a PR to here: https://github.com/pyenv/pyenv
Or maybe https://docs.python.org/3/library/array.html rather than list.
By these standards a 10 year old cpu with a beefy GPU will beat any new cpu as well.
It absolutely does. A simple for loop in the standard Python interpreter will literally take 100x longer than the same thing in a language like C/C++, try for yourself if you don't believe me. CPython is unbelievably slow.
I couldn't reproduce 100x (no optimization flags, otherwise it won't do anything)
Apple clang version 14.0.0 (clang-1400.0.29.102) -> 0m0.347s
ruby 3.1.2p20 -> 0m11.314s
Python 3.8.12 -> 0m19.662s
I thought the differences would be smaller these days
The other languages are also using SDL via their respective interacting-with-C interfaces - what makes Go special here?
https://stackoverflow.com/questions/28272285/why-cgos-perfor...
Even if it is SDL slowing it down, Go FFI being slow is still a real disadvantage compared to the other languages, and you can't just pretend like it doesn't exist in this case.
- With Zig I kept running into compiler bugs, plus no package manager (I’ve vendored SDL and Clap into the source tree)
- C++ I’d occasionally shoot myself in the foot in ways that other languages would have caught, plus no package manager (OS-level package management does an OK job, so long as you don’t mind using old versions, and faffing about with different operating systems acting very differently)
- The pain from Rust was one time where the compiler wanted me to specify a lifetime, and I didn’t understand, so I just spammed lifetime specifiers in various places until it compiled. I’ve been using Rust for a couple of years now and I still don’t really understand lifetimes, but thankfully 99% of the time I can avoid them.
- Nim was a relatively nice language but massively lacking in available libraries (like even parsing command line arguments took me a day just trying to find a library which worked)
- Go is pretty nice, my main pain is the tolerable but constantly-annoying verboseness of error handling (`err := foo(); if err != nil {return err}` compared to rust’s `foo()?`)
- PHP I just hate on a deep and personal level thanks to years of being a PHP4/5 developer. The language is actually mostly-ok-ish these days, but the standard library is still full of frustration like inconsistent parameter orders within a family of functions.
- Python is all-round really nice to write, but the test suite takes like 20 minutes to run, which really messes with my flow-state
" I’ve been using Rust for a couple of years now and I still don’t really understand lifetimes"
Seems like a major pain point.
They do look intimidating to start with, admittedly, and I'll concede that's a negative point for Rust. But it does get better if you practice for a bit.
Lifetimes are necessary when you want to explicitly say "variable X will live just as long as variable Y", or sometimes it's more complex (i.e. you have to specify 2 or more separate lifetimes and then return something that pertains to only one of them) but it's still fairly predictable if you keep it all in your head while coding.
Don't get me wrong I still hate it but it's not as terrible as many people make it out to be. It's hard to get into but also very logical and graspable.
nim: Emulated 600 frames in 0.44s (1367fps)
Perhaps benchmarks should be compared on equal footing, say, with the default release flag or all optimizations turned on, otherwise they're improper.
But really the original author/poster should do some set on his box. I cannot even compile/run all his things. The point of my comment was just to give a reference for how far off impressions can be from build flags. PGO (available to Nim, c++, but maybe not to Rust yet?) is a whole other set of maybe nothing burgers or maybe big improvements.
(But, btw, I could not agree more that all experiments in this entire general space should have various big, bold disclaimers. Over-concluding from these things is rampant.)
For something super jumpy like a simulator, I would find it unsurprising for PGO to make up (or surpass) the difference to Zig in both Nim and C++. 20 years ago there was this ACOVEA [1] project to try to discover great sets of gcc flags that could often find 2X improvements in object code speed for me.
The range from build flags/procedures is often much greater than the supposedly interesting cross-language variation. These things often more measure developer experience/persistence than something intrinsic (and build flags/procedures are only part of that experience/persistence).
There are various debug things you can turn on, such as the race detector, the memory sanitizer, or coverage tracking.
Unfortunately, "just write the performance-sensitive bits in C" is pretty impractical because it only works when you're handing the C routine a relatively small amount of data relative to the amount of work to be done on that data (otherwise the costs of marshaling to C data structures will quickly eat up any gains from processing in C). And unfortunately, it turns out that a whole bunch of code works this way, so the whole bargain of "slow interpreter + easy C extensions" breaks down for a lot of real-world applications, but now we're locked into it.
Processing loops adds a lot of overhead because the interpreter has to make the case jumps after each loop. Figuring out a way to minimize that overhead by using built-in data structures and stdlib library will speed up your code by an order of magnitude.
Don't forget, the built-in types are already running in C.
FFI can pervade an ecosystem making changes (including performance optimizations) to the host language more difficult. It also tends to complicate build and deployment stories. For example, from my Mac I can trivially build a native binary that will Just Work on any Linux system, even one without a libc. Contrast that with Python where I can’t even install popular packages onto most non-Ubuntu Linux distros. And there’s a lot of other things like that which crop up with pervasive FFI.
I’m convinced that the FFI sweet spot is “difficult but possible” and native performance should be good enough 99% of the time.
I'm also wondering if all your #[inline(always)] is slowing things down.
IIRC each of the #[inline] statements was tested and each made a noticable performance improvement. That’s especially true for things like RAM::get() - since the gameboy does I/O by having different chunks of RAM act differently (some address ranges are just RAM, some are hardware controls, some read data from the cartridge, etc) you can replace the hundred-line generic get() with a single instruction if you happen to know that you are looking up one hard-coded address, and that address has no special behaviour.
For the benchmark 600 frames, the Rust emulator calls it 2500049 times, but the Zig emulator only calls it 1808121 times. This is very roughly the ratio between the reported times.
At least one source of this discrepancy is that the Zig emulator doesn't think any sprites are active, but that doesn't account for all of it, and I'm hitting some crashes trying to run it with the display, so I'm probably not going to investigate it further.
(I also hit the same sort of strange Zig compiler unreliability as some other comments have mentioned, where I had to add some logging in seemingly random places to get the Zig emulator to run at all.
And the argument parser just blatantly returns a dead pointer to the ROM path; when I first ran it it just gave "error: InvalidUtf8" until I tracked that down.)
Best wishes
says
# opcache in 8.1 gives a nice speedup (25s to 10s)
Is the 23.7 seconds above using the 8.1 opcache?
[1] https://mypyc.readthedocs.io/en/latest/introduction.html
[profile.release]
lto = true
--- a/py/src/cpu.py
+++ b/py/src/cpu.py
@@ -260,7 +260,6 @@ class CPU:
param = None
cmd_str = cmd.name
self.PC += 1
- self._debug_str = f"[{self.PC:04X}({ins:02X})]: {cmd_str}" zig: Emulated 600 frames in 0.23s (2880fps)
rs: Emulated 600 frames in 0.35s (1691fps)
cpp: Emulated 600 frames in 0.40s (1519fps)
nim: Emulated 600 frames in 0.44s (1367fps)
go: Emulated 600 frames in 1.78s (338fps)
php: Emulated 600 frames in 9.28s (65fps)
py: Emulated 600 frames in 33.10s (18fps)
Applying BS scaling to the above table, Nim would score 1736, only slightly faster than the Rust. Who knows what little flag tweaks in cpp/rs/zig could similarly re-arrange the numbers?
Now, why is -d:lto (or -d:release) not the default build mode? Well, because it takes time and (some) people already complain about compile times and workflow (sometimes). Trade offs abound almost as much as over-conclusion. ;-)
[0] https://github.com/shish/rosettaboy