The point the article is making is this is a "low level" only in the programmer's mind. The point is that for example the mental model you have when programming in python is one mental model and the model you have for C is another. C's might be faster in most implementations, but it is still a mental model and it is a mental model that is somewhat removed from what actually happens when your cpu executes instructions.
People have trained themselves to think in the Python (or js or java or etc etc) mental model while remembering that it doesn't map to what actually happens 100% People have NOT made that leap for C because people still labor under the delusion that their CPU is a fast PDP-11.
In the best case, the compiler finds the instructions I want, and often it does a very good job with details like register allocation.
So the abstraction leaks and inverts like hell. But still, in many cases the C program can still be portable, even if none of the performance tuning is - it targets the wrong pipeline and memory subsystem traits.
When you need to rely on intrinsics, it is different - but I'd say intrinsics are closer to inline asm than C. Saying that as someone who might have ported a DSP library from SSE4 to AltiVec with mostly #define. I know. Also I'm happy it wasn't the other way around!
Google doesn't give me any relevant results
EDIT: Think I found an answer
> "Some very experienced programmer from another company told me about some low-level code-optimization tips that targeting specific CPU, including pipeline-optimization, which means, arrange the code (inlined assembly, obviously) in special orders such that it fit the pipeline better for the targeting hardware."
Answer in comments point to this as a resource:
It basically tries to model statically how instructions travel through the pipeline of a particular CPU, which is useful for finding bottlenecks.
1: https://llvm.org/docs/CommandGuide/llvm-mca.html#timeline-vi...
Could you recommend any books on this topic? Thanks!
Jon also has this book which I seem to remember was fairly good [1]. Don't be put off by the age - uarch on CPU side has mostly been more and more of the same for x86 chips.
0: https://arstechnica.com/author/hannibal/
1: https://www.goodreads.com/book/show/610830.Inside_the_Machin...
What it does is use the scheduling info known to the LLVM optimizers to model how a particular CPU is going to execute your machine code.
I was lucky to start doing this back in the day when 486 was common and the Pentium was brand new. 80486 could do some instructions in parallel if you arranged them very carefully, and Pentium greatly boosted this capability. I used AMD CodeAnalyst (free) back then. I read Zen of Code Optimization by Mike Abrash which explained those particular microarchitectures very carefully. It may still be worth reading to understand how CPUs have evolved, but as this is uarch specific it will not be of great practical use.
The pipelines back then were simple enough to memorize so I spent some boring classes in senior high school plotting various software blitter algorithms on grid paper. Nowadays the superscalar capability is huge and you are better off taking a more statistical approach first - which execution units are stalled or underutilized - and see if you can tweak the instruction mix or find a false dependency that prevents register renaming.
For someone starting out I would recommend studying some smaller Arm chip that has limited superscalar capabilities. Sadly I can't name drop a book that would be a great help in that.
perf related tools on linux: https://perf.wiki.kernel.org/index.php/Main_Page
Didn't read it but it's a well known reference: the dragon book (Compilers: Principles, Techniques, and Tools) 2nd edition has stuff on machine-dependent optimizations (chapters 10 and 11 apparently). Most likely a good read.
The content in the dragon book is the equivalent of a two semester course on compilers, and that's with a prof and TA. Ideally, to get the full benefit of reading this book, you need to reserve a year of your free time after 5pm and be ready to build an optimizing compiler.
It's one of those books that requires your full attention for an extended period of time and you come out the other side a stronger developer, only because it didn't kill you with knowledge.
Edit: I realized now you may be saying the 2 chapters (10 and 11) are most likely a good read to learn about optimizations, not that the entire book is a "good read" in general. Makes sense - I'll leave the comment up, with the disclaimer that I'm referring to reading the entire book cover to cover.
CPU pipelines are extremely dynamic since the slowest actions affecting them (memory reads) are also the least predictable (depends on what ends up in the cache). So it’s usually not worth trying to control them so precisely, but knowing how the first layers work can be good. For x86 the best resources are Agner Fog’s manuals and then the official Intel/AMD ones.
But this may change from compiler to compiler and from version to version of the same compiler. I think you may be better off writing assembly code directly rather than trying to coerce the compiler to do exactly what you want.
I don't see this, I worked on high performance software and everyone is aware that the CPU isn't just a fast PDP-11. This was before 2018, so teaching C programmers about their language isn't the point of the article, everyone who needs to care about performance already knows these things.
This is not the definition used in the article. There is literally a paragraph called "what is a low level language" at the beginning:
> Think of programming languages as belonging on a continuum, with assembly at one end and the interface to the Starship Enterprise's computer at the other. Low-level languages are "close to the metal," whereas high-level languages are closer to how humans think. For a language to be "close to the metal," it must provide an abstract machine that maps easily to the abstractions exposed by the target platform. It's easy to argue that C was a low-level language for the PDP-11.
--
> C maps very well to machine code.
The article's core point is literally a contradiction to this. I'm not sure what to say. Perhaps you ought to read again or provide more arguments as to why you think this it is true.
> A modern Intel processor has up to 180 instructions in flight at a time (in stark contrast to a sequential C abstract machine [..])
> [..] the C abstract machine's memory model: flat memory. This hasn't been true for more than two decades.
> Unfortunately, simple translation providing fast code is not true for C.
--
> "Low level" is relative.
> For a hardware engineer who works on CPU architecture it is of course different, but we are talking from a programmers perspective here.
Point is being made in the articles that programmers do care about cache and pipeline behavior (to get decent performance in intensive parts) or about threading, both of which are transparent to C. And also that languages otherwise seen as "high-level" (usually because of memory management) sometimes have aspects that map better to these features (hence are lower level than C in some other aspects).
Then say C fails this test of being low-level:
> A modern Intel processor has up to 180 instructions in flight at a time (in stark contrast to a sequential C abstract machine [..]) > [..] the C abstract machine's memory model: flat memory. This hasn't been true for more than two decades. > Unfortunately, simple translation providing fast code is not true for C.
None of this is true for assembly either. To my knowledge neither x86 nor ARM expose primitives to control runtime instruction parallelism nor cache. You just have to know what that looks like and how to adjust that implicitly as best you can (still with no guarantees). Maybe a CPU architecture like Vulkan would work where there’s low-level primitives and vendor-specific plugins but I don’t know. GPU programming has gotten significantly harder and error-prone with Vulkan. Additionally, game-programming doesn’t need pixel-perfect behavior while you do typically want that out of your CPU (yes it’s slightly inaccurate with GPU compute advancements, but that’s still significantly more expensive as a development target and reserved for problems where the value is worth it)
(I mean, the point that "it's higher level than you think" might be a reasonable one to make. But arguing about the definition of "low level" may not be the best way to make that point.)
By the way: what’s your take on VLIW architectures? Possibly with the currently hardware level optimizations moved to software?
Re VLIW architectures: I don't know enough to have a take about them specifically. But I fear moving the optimizations from hardware to software, because it's going to be really hard to optimize as well as Intel/AMD/Arm do, and it's going to be really easy to mess something up. (I don't write assembly if I can help it - I let the compiler writer do it. I don't manage memory if I don't need to. And so on.)
I can see some people might benefit from this, but I suspect that it's bleeding-edge people only. Most of us won't gain from going this route.
Note well: Just my take. Your mileage may vary.
And of course if you’re writing EFI code sometimes you haven’t turned the RAM on yet and cache is what you’re executing out of.
No. The article's core point is that machine code maps poorly to what the CPU does.
Modern CPUs expose no low level access to the way they execute code.
C does map perfectly well to the abstractions exposed by x86 CPU. By this definition, C is a low level language. The author himself states that modern processors are fast pdp-11 emulators. This is exactly the "abstractions exposed by the target platform".
If C is not a low level language by that definition, then nothing is. Assembly uses the same PDP-11 abstract machine model that C does. Hence the parents point that this definition is useless.
Not sure how to say it but they state the exact opposite! Modern processors are not pdp-11 emulators and people are led to believe that because of immense work done by compilers (hence not low level).
> Assembly uses the same PDP-11 abstract machine model that C does.
Not really (assembly has things for SMP memory consistency and cache management), but even that is irrelevant since the article's point is a call to stop shoehorning modern processor capabilities onto that old sequential model and designing an actual low level language with full access to these capabilities (explicitely and not implicitely as is currently done for eg ILP).
The author very clearly states that modern processors _are_ presenting a PDP-11 interface. The author argues that they shouldn't be doing that, but all modern processors are still presenting a pdp-11 like abstract machine:
> The root cause of the Spectre and Meltdown vulnerabilities was that processor architects were trying to build not just fast processors, but fast processors that expose the same abstract machine as a PDP-11.
Depending on what you want to do. One can also go a level lower with machine code, or physics as how some quantum computers are coming on the scene now days.
Again, that depends. If you're a compiler engineer, it's likely you might disagree with that statement, because your job is to create passes that change that C into machine code that barely resembles the original source but performs the same operation, faster–except that "same operation" isn't even defined in terms of the machine code, it's defined in terms of an abstract machine that doesn't exist outside of the C standard. Does C really map well to machine code when it reorders your statements and enregisters your variables and unrolls your loops and inlines your functions? I think that really depends on where you're sitting on the ladder of relativity.
Language levels of abstraction:
1. Machine code
2. Assembler
3. A high-level language
C does allow low level things, e.g. access of machine registers or embedding of assembler or machine code, but it is also capable of quite high level of abstraction.
Some of 3-rd level languages allows higher level of abstraction, or hide low level abstractions, e.g. memory management, stack, byte order, bits per number, etc., hover none of them improves programmer productivity by an order of magnitude.
C isn't converted into assembler language unless you ask the compiler to. For example, LLVM lowers IR AST structures into target specific machine code. Assembler is never involved.
ASM is just a human readable representation of machine code for a particular ISA. So in your mental model, C and ASM exist at the same level.
Further, Java bytecode would exist above the the machine code level (again, alongside C then, since the VM is compiled to machine code) and Java on top of that.
It just doesn't make any sense to define languages like this. I don't see any benefit, and I don't even see it as being correct.
[1] https://www.oldcomputerbooks.com/pages/books/R504/james-mart...
[2] I can't find a good example, but I remember seeing ads for "4GL" tools in Dr Dobbs regularly in the mid-90s.
That depends on the compiler. GCC, for instance, most certainly emits assembler code as part of its standard compilation process.
"The SSA form is also used for optimizations. GCC performs more than 20 different optimizations on SSA trees. After the SSA optimization pass, the tree is converted back to the GIMPLE form which is then used to generate a register-transfer language (RTL) form of a tree. RTL is a hardware-based representation that corresponds to an abstract target architecture with an infinite number of registers. An RTL optimization pass optimizes the tree in the RTL form. *Finally, a GCC back-end generates the assembly code for the target architecture using the RTL representation.* Examples of back-ends are x86 back end, mips back end, etc."
Then, there's the official GCC documentation[2]:
"Compilation can involve up to four stages: preprocessing, compilation proper, assembly and linking, always in that order. GCC is capable of preprocessing and compiling several files either into several assembler input files, or into one assembler input file; then each assembler input file produces an object file, and linking combines all the object files (those newly compiled, and those specified as input) into an executable file."
You can also just run GCC -- either with --verbose or --save-temps -- and observe what the compiler toolchain is doing.
[1] https://en.m.wikibooks.org/wiki/GNU_C_Compiler_Internals/GNU...
[2] https://gcc.gnu.org/onlinedocs/gcc-11.2.0/gcc/Overall-Option...
> SSA GIMPLE is low level GIMPLE rewritten in SSA form.
I’m by no means a compiler internals expert, but to my knowledge there’s no translation from RTL to assembly. The compiler is directly emitting the machine code directly without first emitting assembler. You have to ask the compiler to serialize to assembly explicitly.
> several assembler input files, or into one assembler input file; then each assembler input file produces an object file,
That doesn’t sound right. Seems more like an abstract description. You can’t dump assembly out of an object file either. You have to do a conversion from an intermediary form to get back out a form of the assembly.
I think at this point it’s splitting hairs but assembly is not used as an intermediary at any point, even for generating machine code.
You asked me for a citation. I provided two, as well as a means by which you could see for yourself that what I am saying is correct.
Believe what you want, but all of the GCC documentation, as well as the output of the compiler if you run with --verbose or --save-temps supports what I have described of GCC's operation.
The compiler internals manual[1] also describes the backend machine descriptions, and how they are used in detail:
"There are three main conversions that happen in the compiler:
1. The front end reads the source code and builds a parse tree.
2. The parse tree is used to generate an RTL insn list based on named instruction patterns.
3. The insn list is matched against the RTL templates to produce assembler code."
> hover none of them improves programmer productivity by an order of magnitude.
I disagree. I can write efficient SQL statement in 2 minutes, but implementing efficient multiple "table" join with spill to disk took me much longer than 20 minutes.
It's like saying Windows 11 and OS/2 are the same because they are both Operating Systems.
In this context, the question is more, are Lisp or Haskell, or Python 10 times as productive as C? I used to think so, but now I seriously doubt it.
I mean, the reason I’m so much faster at writing small Python programs than I am at writing equivalent C programs, is because Python’s standard library is so damn huge. This is not a case of the language being more productive, this is a case of the work already being done.
Your portability point mostly stands, though.
«A Compiler is a software that typically takes a high level language (Like C++ and Java) code as input and converts the input to a lower level language at once.»
«Assembler A translator from assembly language programs to machine language programs.
Compiler A translator from “high-level” language programs into programs in a lower-level language.
Transpiler A translator from high-level language programs into programs in a different high-level language.»
«The compiler is software that converts a program written in a high-level language (Source Language) to low-level language (Object/Target/Machine Language).»
This the best possible outcome, though -- since they used up these kind of "biggest classification that doesn't sound totally silly" labels in the 70's and 80's or whatever, we don't have to worry about classifying languages anymore, which was not really all that productive to do anyway.
Although I do wonder what level Verilog should be considered.
For example, surely languages like Typescript are "level 4" as you cannot manipulate memory directly but only use the languages higher level type structures.
https://en.wikipedia.org/wiki/Fourth-generation_programming_...
They considered things like excel 4GL: You program some formulas and excel magically finds out what the necesarry steps are to do the computation.
A lot of vendors created graphical programming languages and declared them 4GL. The term mostly died because nobody could stop laughing.
The first 3 generation of languages were powered by higher levels of abstraction, which were possible because of better machines.
However, further increase in level of abstraction did not make developers an order of magnitude more productive, except for niche areas. Switching to a better development process (Scrum) can achieve order of magnitude improvement in developer performance, while switching of programming language cannot.
Anybody can claim that their product is a next gen revolutionary market shaking breakthrough. It's the self-promotion, not a science.
And I still have to review all of them.
How is that "4th level abstraction" whatever the current usage of the term?
Transpilers are compiling from one 3rd level language into another 3rd level language.
Code generators or macros are receiving data and producing code in a 3rd level language.
4th level language receives plain text explanation of a goal and produces code in a 3rd level language. AI based tool is the only tool which is capable to do that, because such process requires understanding of the goal, not a mechanical transformation of input into output. It's possible to implement such AI in Lisp, Prolog, or using ML. Copilot is an example of such AI.
Such an explanation would read like this:
I require some glue code so the system our company uses to pass messages between our services, can talk to the system of the company we talked to 2 days ago. Also, there should be a web interface or something to monitor it.
A function that queries the user from the database by its surname apt-get install foo bar baz
Unfortunately, this is not a general-purpose language, but apt + shell is.
I issue an imperative command to apt-get, and it tries to execute it.
It has no knowledge of why I want foo bar and baz installed, nor can it decide if these 3 programs make sense for what I am trying to do, or if there is a better way to do it (maybe there is a package "bazinga", that can replace the foo-bar-baz stack?).
A goal-aware package manager would look like this:
super-apt "i need the system ready to function as a webserver, with a high availability database and backup capabilities"A typical package manager is already goal-aware, like `make`. It's input DSL is far from English, but it's not hard to translate your English sentence into «apache postgress rsync» by matching text to package descriptions.
Package manager requires a solver for dependencies, constraints, and conflicts, which is far from trivial. For example, Fedora(RedHat) switched from in-house solver for yum/dnf to zypper SAT solver (libzypp) developed by SuSE/Ximian/Novell.
Unfortunately, package managers are DSL, not a general purpose language.
If you are asking what level of abstraction is used by Forth, then the answer is: 2nd. It directly exposes stack, memory, and machine code and provides a thin wrapper on top of that. Developers can build higher levels of abstraction then, of course. It's possible to do an object-oriented programming in machine code, because compiler can.
The trickier question is: What level is Lisp? Is it assembler for Lisp machine (because of opcodes like car, cdr, etc.)? Or, maybe, it is a 4GL because of it advanced meta-programming possibilities?
So, it's expected to see an order of magnitude improvement in development speed, on average, when jumping from machine code to asm, or from asm to a 3rd level language. Forth doesn't improve speed of development by an order of magnitude comparing to asm, because of the steep learning curve and low level stack management. I tried to learn Forth multiple times, but still cannot program in Forth freely, which makes it unique among 20+ other programming languages I know.
Chuck Moore created Forth to improves his productivity versus conventional tools and there is some anecdotal evidence that it worked for him and those that worked with him, particularly where direct hardware control on new hardware could be interrogated and validated via the interpreter rather than an Edit/Assemble/Link/Test loop.
[1] - https://www.discoverbooks.com/Forth-The-Fourth-Generation-La...
We could say that a temperature is "freezing" if it is under 0C, and "boiling" if it is over 100C. But it's hard to nail down "cold" or "warm", as any group sharing a thermostat will tell you.
You can easily place arbitrary assembly commands or memory values in C code. Plenty of embedded codebases are littered with asm blocks in performance-critical areas. It's hard to see it as a high-level language in the age of javascript/python, but maybe the overton window has shifted.
I know in college, a lot of my profs referred to C as a 'mid level' language (or by others as a 'systems' language), C# 1.2 and Java were 'high level'. They really only considered ASM as a low level, but that was almost 20 years ago.