News for hackers. Stuff that matters.
Re:Pallene: A statically compiled companion language for Lua by Rochus on Monday July 29, 2024 @08:55AM (#41100088)
The problem is actually FNEW. If you create your function on top-level and leave it as is then FNEW is only used once. But if you require e.g. nested functions, then each call to FNEW will terminate the trace. My measurement with the Smalltalk and SOM implementations using LuaJIT showed that the benchmark essentially remains in the interpreter most of the time for that reason.

> it seems plausible to me that it would be more convenient to reimplement a lua function in pallene rather than in c

If just Pallene would be more similar to Lua. But essentially it's a different language which just resembles Lua a bit. So you could use any language compatible with the Lua C API if you don't like C (instead of learning Pallene). You have to rewrite the performance critical parts anyway. Personally I prefer LuaJIT, since everything is Lua, and if you know enough details and do measurements, the performance is no longer a surprise.

> how good or bad the performance of the code generated by the pallene compiler is

There is no reason to assume that Pallene is faster than LuaJIT if things like the ones mentioned above are considered.

Re:Pallene: A statically compiled companion language for Lua by kragen on Monday July 29, 2024 @09:37AM (#41100342)
that must have been a huge disappointment when you ran into that problem with som! are you planning to write your own jit compiler, or is there still a way you can leverage luajit?
Re:Pallene: A statically compiled companion language for Lua by Rochus on Monday July 29, 2024 @10:17AM (#41100661)
I ran into stability problems with the Oberon+ FFI which I was not able to solve with the LuaJIT bytecode generator; so I switched to Mono, which even has a much better debugging infrastructure, and for speed I also implemented a C transpiler, the result of which is very close to the native C++ implementation. SOM was mostly an effort to find out how close we can come to Cog with a LuaJIT backend, as a follow-up to my Smalltalk LuaJIT adventure.
Re:Pallene: A statically compiled companion language for Lua by kragen on Monday July 29, 2024 @11:49AM (#41101359)
i see!

hmm, are you going to have to switch to .net core now? or are mono users succeeding in banding together to handle security updates in precisely the way python 2 users have failed to?

Re:Pallene: A statically compiled companion language for Lua by Rochus on Monday July 29, 2024 @12:42PM (#41101743)
I'm not using the dotnet framework, just the clr, and for the use cases covered by Oberon+ security is of little concern. The project is still regularly updated, see https://github.com/mono/mono/commits/main/.
Re:Pallene: A statically compiled companion language for Lua by neonsunset on Monday July 29, 2024 @01:06PM (#41101924)
Please don't use this Mono flavour as it is still outdated. If possible, please use the Mono runtime that lives in https://github.com/dotnet/runtime/tree/main/src/mono

Ideally, CoreCLR will give you much better performance, ability to AOT compile your binaries to native executables and will remove the need for multiple back-ends.

Re:Pallene: A statically compiled companion language for Lua by Rochus on Monday July 29, 2024 @02:21PM (#41102523)
These are two different Mono implementations. The one maintained by MS has a different focus. I'm mostly interested in a lean, independent build and many different targets (e.g. Apple M1). The "much better performance" of CoreCLR is a fairy tale (see e.g. https://www.quora.com/Is-the-Mono-CLR-really-slower-than-Cor...). Actually, Mono supported AOT even before the CoreCLR (but I don't need it, because I have a to C transpiler which produces cross-platform near-native performance).
Re:Pallene: A statically compiled companion language for Lua by kragen on Monday July 29, 2024 @02:40PM (#41102651)
this is fantastic information, thanks!
Re:Pallene: A statically compiled companion language for Lua by neonsunset on Monday July 29, 2024 @03:12PM (#41102882)
Unfortunately, there are a few issues with this quora post. First it is outdated - .NET is on an annual cadence and statements true for 3.1 or 5 have no impact on user experience today, second it focuses on edge case differences with log scale rather than the full picture. Also note that compiler improves significantly with each version - look at the difference between 3.1, 5 and 6. The differences between 6 and 8 are way more significant (DynamicPGO light-up, physical promotion for structs, etc.). And third there is more to the story than microbenchmarks. Again, there is a huge rift in performance between Mono and CoreCLR, and there are classes of optimizations Mono does not and will not do, like devirtualization. All optimization work goes into CoreCLR for multiple ISAs. Mono does not support AVX2 and AVX512, which core routines in .NET CoreLib rely on if available. It doesn't have a fancy if-conversion pass to merge branches and comparisons that CoreCLR does for ARM64, write barrier optimizations, etc etc.

This is not a myth but what anyone who works closely with .NET or is a part of .NET teams will tell you.

For example, look at https://aka.ms/aspnet/benchmarks select mono tab and pick coreclr and fortunes - you'll notice that Mono is more than five times slower than CoreCLR. And it is the Mono flavour which also sees performance improvements for the features that .NET adds or optimizations focused on targets served by Mono.

Also, the independent build is not independent - it limps along Unity Technologies contributions until Unity finishes its move to CoreCLR, which they are in the process of.

Re:Pallene: A statically compiled companion language for Lua by Rochus on Monday July 29, 2024 @04:11PM (#41103334)
The post is good enough, using recognized methods and allowing reproducability. It considers different CoreCLR and Mono versions, and there is no reason to assume big performance differences since. The raw data is referenced, you can use a different scale if you prefer, but the information is the same. If you are interested, take a look at the Are-we-fast-yet benchmark suite to see that it is well defined, not just a bunch of random micro benchmarks, but also includes a few larger programs (e.g. to exercise the GC). And again: this is about the CLR, not the framework. I'm not interested in the .NET framework implementation. The build works well on all platforms without any dependencies and without downloading and compiling several gigs of stuff.
Re:Pallene: A statically compiled companion language for Lua by neonsunset on Monday July 29, 2024 @04:20PM (#41103399)
I am talking about CoreCLR, which has nothing to do with .NET Framework. It has tangentially something to do with .NET's standard library but its interaction with it is the same as the one Mono has. It just executes CIL assemblies, maybe AOT compiles them if you ask build system for it.

The statement "there is no reason to assume big performance differences since" is naive and ignores the performance work that went into releases 7 and 8, particularly DynamicPGO which enables the whole class of new optimizations. It also ignores the steady, significant improvements in versions 3.1, 5 and 6. These are annual jumps in performance on code that had otherwise no changes.

I caution other readers to take extra grain of salt and perform their own benchmarking if they do not believe my statements. Luckily, the numbers will be very easy to interpret with difference occasionally being as much as an order of magnitude in favour of CoreCLR.

Mono is easier to embed which justifies select scenarios of its usage, but if there is no such specific requirement, then you should use CoreCLR. Doing otherwise means harming your project and its users in the desire to be stubborn.

Re:Pallene: A statically compiled companion language for Lua by Rochus on Monday July 29, 2024 @04:29PM (#41103473)
If you want to convice me, try running the referenced code on the most recent Mono and CoreCLR version and relevant architectures, and collect/report the data in an accessible way. I'm only interested in reproducible measurements, not in opinions.

> Luckily, the numbers will be very easy to interpret with difference occasionally being as much as an order of magnitude in favour of CoreCLR. ...

Thanks for the sales pitch.

Re:Pallene: A statically compiled companion language for Lua by neonsunset on Monday July 29, 2024 @07:53PM (#41104807)
It was interesting to look at how .NET deals with arbitrary assemblies from something that isn't C# or F#. In the end, I did not bother with manually linking .dll files for ILC so there will be only JIT results. In general, I do not think the CIL back-end for Oberon produces good code.

It is compiler-unfriendly, ignores functions provided by .NET's CoreLib (available both under Mono and CoreCLR) to use its own poorly performing variants of them, makes everything virtual and etc. It does silly things like makes all references to a string literal into an array allocation ("test".ToCharArray()) to just do a comparison, then also treats such arrays as zero-terminated pointers despite, well, .NET arrays having length in them included already, and more.

Overall, a disregard what CIL, CLI and CTS are and what they offer. What are you using .NET for if not for writing truly portable C-like? There are already pointers and manual memory management, the lowering strategy could have been much closer, and a lot of suffering is very much self-imposed.

And the benchmark initially ran for so little time that it would give advantage to pure interpreter based runtime as long as it did very little, or natively compiled binaries. Surely your average program runs more than 200ms?

Anyways, you can find these and a couple other notes with data and charts here:

https://gist.github.com/neon-sunset/900bd310dc53dd88cbeddb51...

Geometric Mean (Mono): 1612026.37 microseconds

Geometric Mean (.NET): 847482.65 microseconds

Relative .NET advantage: 190.21%

I did not have time to look closer in benchmarks with 1:1 numbers, but if you note things like JSON handling which is compiler sensitive, you'll note that the difference there was the biggest.

Re:Pallene: A statically compiled companion language for Lua by Rochus on Monday July 29, 2024 @08:54PM (#41105132)
Apart from the unnecessarily arrogant remarks, which by the way are completely irrelevant since the same code runs on both measured VMs, no matter how you find it, I find the result interesting and thank you very much for your effort. I had never run CoreCLR on Apple M1 before. Apparently there was indeed a further improvement on this architecture, although still the overall difference is not that dramatic. If I analyze your numbers the same way I did before, I get a geometric mean factor of 1.8, which would almost make it worth switching to CoreCLR, at least on Apple M1. But there are other critical requirements, e.g. that CoreCLR does not support all the platforms I need, and that the debugging support, as far as comparable at all, requires a completely different integration. Just for the speed-up on Apple it's not worth switching. As soon as I find some time, I will try again on x86 and x86_64 and see how it looks there in comparison to my 2021 measurements.
Re:Pallene: A statically compiled companion language for Lua by Rochus on Tuesday July 30, 2024 @06:08AM (#41107635)
Here are some comparisons to my 2021 measurements.

- Mono 5 and Mono 6 on Windows 10 x86 achieved about the same performance (factor 1.04)

- Taking Mono 5 as a reference on Windows 10 x64, CoreCLR 3 had about the same performance, CoreCLR 5 was factor 1.3 faster than Mono 5, and CoreCLR 6 was factor 1.2 faster than Mono 5.

- The measurements done by neonsunset on Mac M1 show that CoreCLR 9 is factor 1.8 faster than Mono 6; therefore CoreCLR 9 (M1) seems to be around factor 1.4 faster than CoreCLR 5 (x64).

Re:Pallene: A statically compiled companion language for Lua by neonsunset on Tuesday July 30, 2024 @06:26AM (#41107719)
The issue of poor lowering strategy remains. The back-end should be using structs, pointers and manual memory management if this is what it does with C. C# supports full set of features to make it possible including (now) raw function pointers. Just no C macros, sorry.

It should not even be close to Mono in performance. It should be close to C, given correct back-end output, which it isn't. At this point, outputting C# would have produced much better results that do not defeat compiler optimizations, where CoreCLR performs rather well despite the CIL quality, not because of it. And even then, you can easily see that either computationally intensive or logically complex benchmarks show the biggest difference as they stress out compiler capability the most. In any case, the option to target C# instead would result in guaranteed correct CIL form, with loops written in a way that does not occasionally break loop recognition, does not introduce unnecessary memory and abstraction overhead and does not have the issues with the basics like that one with string comparison that allocates each time for no reason.

Also note that compiler improvements are generic - most affect any platform supported, with platform-specific peephole optimizations responsible for cleaning up the codegen and eking out the last %. If-conversion pass for ARM64 is going to be an exception, but on the other hand x86_64 offers wider vectors which .NET makes use of.

Re:Pallene: A statically compiled companion language for Lua by Rochus on Tuesday July 30, 2024 @08:31AM (#41108475)
You are still clinging to the same misconception. I want to measure the CLR, not the frontend output. The CLR is by definition a "Common Language Runtime", i.e. it makes the promise to execute applications written in multiple high-level languages without the requirement that these applications have to take into consideration the unique characteristics of the specific CLR or environment. There is no requirement in ISO 23271 whatsoever that the CIL fed to the CLR must meet other requirements than the ones stated in the standard. There is especially no requirement in the standard that the bytecode fed into the CLR must be optimized or meet certain patterns so that loops are recognized. My code is perfectly compliant with the standard.

The CLR is essentially a JIT processing a well-defined intermediate language. A JIT is all about optimization. In contrast to a static compiler, a JIT can also consider specific system and runtime information for this purpose. Mono is very good in both static and dynamic optimizations, and it is especially good in handling bytecode from other than the included C# compiler. Microsoft, for some reason, seems to have decided to move a lot of optimizations to their new C# compiler infrastructure, which also heavily uses inside knowledge of the CoreCLR, and generates patterns which the latter is looking for (e.g. for loop detection and the like). It is therefore not surprising if code compiled with Microsoft's own C# compiler runs faster on the CoreCLR than other code.

However, I am only interested in the original purpose of the CLR. I don't want to have to use .NET or Microsoft's compiler framework in order for the CLR to achieve good performance. That is why I deliberately use unoptimized code for the performance measurements, which in particular does not take into account any of the rules that Microsoft has introduced outside of the above-mentioned standard.

Your arguments would be relevant if different C# compilers + CLR were to be compared with each other, or to my Oberon+ compiler. That is a different use case. As a language developer and not equipped with the resources of Microsoft or the CoreCLR team, I have been able to save a lot of effort by profiting of the CLR standard and of Mono in particular, as forseen by the original purpose of the CLI/CLR. My measurements and claims represent this original purpose and use case.

If I had supplied highly optimized bytecode instead, I would - besides my much larger effort - to a large extent be measuring the compiler frontend (or its consideration of the many, sparsely documented rules that the bytecode must take into account beyond the standard in order for CoreCLR to run optimally), not the CLR.

That said, if you want to immortalize yourself in the Are-we-fast-yet community, you could contribute a C# implementation of the benchmark suite. This would be a good way to measure the influence of the frontend on CLR performance, which would also be very interesting.

Re:Pallene: A statically compiled companion language for Lua by neonsunset on Tuesday July 30, 2024 @10:00AM (#41109288)
I won't - already spent enough time on this and don't have more. My points were only that repurposing C emitter to target C# as well, with the same structs/pointers/memory management is likely both easier and bound to produce massively faster performing solution than something that has deliberately compiler-unfriendly IL. But even if you target CIL, intentionally using incorrect structures or the ones that are different to C back-end, especially around string comparisons, sounds like ignoring what the platform can offer. You can keep discounting my suggestions as something that is outside of ECMA spec and to be ignored, which it isn't. At least the cases like this one are few and far between and serious projects like IronPython3, ClojureCLR or Godot use appropriate runtime.
Re:Pallene: A statically compiled companion language for Lua by kragen on Tuesday July 30, 2024 @08:52AM (#41108622)
that's exciting data!
Re:Pallene: A statically compiled companion language for Lua by kragen on Monday July 29, 2024 @06:10PM (#41104257)
i notice that debian still doesn't have a .net core package. do you know what the obstacle is? is there a hidden licensing problem, or are the debian developers having trouble getting the released source to compile successfully? they do have a mono package so it can't be that they're morally opposed to c#
Re:Pallene: A statically compiled companion language for Lua by neonsunset on Monday July 29, 2024 @07:39PM (#41104745)
It should be available depending on your source list? sudo apt install dotnet-sdk-8.0
Re:Pallene: A statically compiled companion language for Lua by kragen on Tuesday July 30, 2024 @08:57AM (#41108661)
https://packages.debian.org/search?suite=all&searchon=names&... says, 'your search gave no results', showing that nothing similar to dotnet-sdk-8.0 exists in debian. what i'm asking is why not: is it a licensing problem or a difficulty in building?

i'm not interested in whether some fly-by-night third-party software provider offers a repository of debs

Re:Pallene: A statically compiled companion language for Lua by neonsunset on Tuesday July 30, 2024 @11:48AM (#41110346)
Most distro feeds do have it, hell even on FreeBSD it's just `pkg install lang/dotnet`. I guess Debian is special just like that (not that I care), there can't be a licensing issue because it's all MIT and can be built (takes a lot of time though) from a monorepo https://github.com/dotnet/dotnet. I must admit the way you pose the question strikes me as strange. What are you actually trying to say?

In any case, there's an official guide: https://learn.microsoft.com/en-us/dotnet/core/install/linux-...

Re:Pallene: A statically compiled companion language for Lua by kragen on Tuesday July 30, 2024 @12:32PM (#41111034)
thanks! yeah, i'd checked out the official guide previously, but it doesn't explain how to build from source, just how to download prebuilt binaries from some fly-by-night third-party software vendor website. that's part of what made me suspicious!
Re:Pallene: A statically compiled companion language for Lua by neonsunset on Tuesday July 30, 2024 @01:23PM (#41111763)
Building the entire Roslyn + runtime + additional packs + sdk installer and other bits is something that makes little sense to do for pretty much anyone but specific distro maintainers that have policies that require built-from-source only packages, organizations or select individuals that just want to do it because reasons.

Otherwise it takes way too much time as it means compiling multiple large projects (surely you're not always building LLVM or Chromium from source?), this isn't really something official documentation for the regular user needs to concern itself with but the documentation in the respective repository.

Re:Pallene: A statically compiled companion language for Lua by kragen on Tuesday July 30, 2024 @01:50PM (#41112099)
i don't want to become dependent on software where in theory it's free for me to study, modify, and redistribute modified versions of it, but in practice doing so is far too difficult to be worthwhile. it sounds like you're saying .net core has slid pretty far down that slippery slope?

debian does of course always rebuild llvm and chromium from source

Re:Pallene: A statically compiled companion language for Lua by neonsunset on Tuesday July 30, 2024 @02:14PM (#41112361)
Have LLVM, GCC or, say, Firefox or Chromium slid pretty far down that slippery slope? It might seem you're just looking for an excuse to make a straw-man of, which is not a productive way to approach whatever it is that grinds your gears.

You can build .NET (like the entire thing) on your machine (and the readme does a good job to guide you through that), it's just not a very practical thing to do for a regular user. Exact same reason applies to OpenJDK which is why there are usually community-maintained builds of it like Adoptium (besides addressing support-contract trap by Oracle which some of the "first-party" Java SDK builds are).

In the case of Debian, it is the target audience for the use of dotnet/dotnet VMR. Why they are not using it is something I don't care about though. My choice would be something saner for daily driving like Fedora anyway.

Re:Pallene: A statically compiled companion language for Lua by kragen on Tuesday July 30, 2024 @06:13PM (#41114634)
firefox and chromium yes, llvm is about halfway there, and gcc has fought its way a significant distance back up that slope since the first time i built it almost 30 years ago