You use a compiler to abstract away the CPU. Because of that, there is no “run as written”.

Simple example: if you declare a structure that is 14-bytes in size, declare variables foo and bar of that type, and then write

   foo = bar;

C must behave as if 14 bytes are copied from the memory where bar is stored to the memory where foo is stored, wit two observable states: foo either has its old contents or the new ones. Chances are very high that your CPU cannot do that.

The compiler will instead do something different. It might call memcpy, it might copy 8 bytes first, then 4 more, then the last 2, it might copy the first 8, then the last 8 (copying the two center bytes twice), etc.

Similarly, if you declare an int named i and do

  i += 1;

There’s no guarantee that will translate to a single instruction (what if your CPU has 8-bit registers, but your compiler chose to have 16-bit ints?). Even if it does, does “as written” mean an increase instruction or an addition? Your CPU might have neither, but only subtraction and negation.

Also, a literal interpretation of “as written” requires spilling the result of every statement back to memory; a strict literal interpretation even forbids using registers (if you don’t write “copy i into a register, add 1, write the result back to i”, the compiler can’t do that if it has to run your code “as written”)

But yes, it would be awesome if somehow the tooling or the compiler could report some of these things in an easy to visualize not-super-verbose way.

Unfortunately, that isn’t possible. Modern languages and their compilers are just too far abstracted away from modern hardware.

Being caught in the middle of an instruction, like all your examples, is much less surprising than when something never happens at all.

> Also, a literal interpretation of “as written” requires spilling the result of every statement back to memory; a strict literal interpretation even forbids using registers (if you don’t write “copy i into a register, add 1, write the result back to i”, the compiler can’t do that if it has to run your code “as written”)

If you don't make a pointer to a variable then it's allowed to be in a register, isn't it? There's even a 'register' keyword, however obsolete it may be.

I think it’s only less surprising because it happens more often, so that ‘everybody’ knows about it, not because it is inherently less strange.

As examples of merging, if you do

  s = sin(x)
  c = cos(x)

would you find it surprising if that generated a single sincos instruction if the CPU has one?

If you do

  typedef struct s { char a; char b; }
  s c;
  […]
  c.a = 3;
  c.b = 6;

do you find it surprising if the compiler generates a single instruction that is equivalent to (exact code will vary according to the CPU architecture, and this may need a further cast to avoid undefined behavior):

  *(short *)(&c) = 0x0603;

I thought compilers ignored those words when optimizations are turned on, compilers reorganize and rewrite your code.

Using an interpreter seems to be the best option to abstract away the CPU. Write the interpreter once, get enough eyeballs onto it, test it 'till death and use it for predictability.

Even if you write assembly these days, there is all sorts of ambiguity around. Even the underlying hardware we use can fail.

I used to believe that computers did as they're told, they could do no mistakes, all mistakes were human-made. Now I see that there is no such thing as error-free. We are basically living on chance, much is uncertain.

I trust my eyes and ears more and more instead of the machine.

No "run as written"? Tell that to interpreted C++.

> You use a compiler to abstract away the CPU. Because of that, there is no “run as written”.

Um, no. I use a compiler to generate code that would otherwise be very tedious to write. I do not expect it in any way to abstract away the CPU. I do not want it to abstract away the CPU. If I did I would write Java or python. I'm using C++. I am very aware of the CPU. What I'm not aware of is whatever silly transformations the optimizer thinks it can do to my carefully crafted program because I didn't read page 783 of the spec.

You want a macro assembler. C is not a macro assembler. C was never supposed to be a macro assembler. It was designed as a cross platform language, which fundamentally means you are programming to an abstract machine, which necessarily means that you cannot have an macro assembler [0].

I can, however, see an argument that people want a C like language which is a macro assembler. I am lost again when people start asking for a C++ like macro assembler. The features that C++ brings over C are not concepts that exist on any mainstream CPU.

[0] Unless you decide to build a concrete machine (either in bare metal, or as a VM).

C was originally a macro assembler! This new mutation of C is just that, new. I don't mind optimizations as long as it doesn't remove intent, which is usually clearly indicated in the source, standards be damned.

How far back to you want to go?

There is no definitive start point for C, as it grew fairly organically out of B and began as a living language. However, a reasonable approximation for C 1.0 is The C Programming Language (Kernighan and Ritchie, 1978) [0], commonly known as K&R C.

The language they describe is much simpler than modern C. Notably for this discussion, it contains no synchronization mechanisms.

There is nothing in that book that suggests C was a macro assembler. On the contrary, its description of C precludes it from being a macro assembler for the reasons I talked about in a parent comment:

> Although C matches the capabilities of many computers, it is independent of any particular machine architecture, and so with a little care it is easy to write "portable" programs, that is, programs which can be run without change on a variety of hardware. . It is now routine in our environment that software developed on UNIX is transported to the local Honeywell, IBM and Interdata systems. In fact, the C compilers and runtime support on these four machines are much more compatible than than the supposedly ANSI standard versions of Fortran.

> Because the data types and control structures provided by C are supported directly by most existing computers, the runtime library required to implement self-contained programs is tiny. On the PDP-I1, for example, it contains only the routines to do 32-bit multiplication and division and to perform the subroutine entry and exit sequences.

Notice the way this paragraph frames the issue. Since the features of C are supported by most machines, it can be implemented with a small runtime library. It does not say that C must use those features directly. In fact, it explicitly anticipates that C could provide some of its primitives in ways other then directly leveraging a corresponding capability of the hardware. The feature here is efficiency and a small runtime.

[0] https://1lib.us/book/2660358/b1a4cf

I'm not sure where you got that idea. C was never ever a macro assembler, and its antecedent language (B[1]) is actually even more abstracted away. C was a slight lowering, not a raising, of hardware abstraction.

If C were a macro assembler the following things would be true of it, most likely:

  - types would have fixed bit sizes (ie. uint32_t would be the norm)
  - you'd be able to do various 'normal' things for assembly like rotate left and right.
  - inline assembly would probably be.. you know.. standard instead of something that compiler vendors only started putting in in like the 80s.

You might be thinking of the fact that C++ was originally sort of a Macro C (aka Cfront)?

[1] Ever wondered why `int` is so fuzzily defined in C? It's because in B there was only one type and it was always word sized. That's also why various things are typed `int` by default if you don't specify.

Another extremely normal thing assembly allows you to do, but C doesn’t: access carry and overflow flags.

Not all CPUs have carry and overflow flags. For example, MIPS, or SPARC, or the new RISC-V.

really relevant to this discussion, though, the pdp11 did.

Another (modern) example: SIMD instructions.

> I'm not sure where you got that idea

History: C was essentially PDP11 macro assembler. There are so many assumptions in C that basically came from the PDP11. C was originally a very utilitarian language to make writing unix easier. That it became massively popular was more of a happy accident.

I'm sorry, but this is just a gross mischaracterization of what a "macro assembler" is. I agree it was utilitarian and made to make unix easier and that many of its core assumptions were based on the platform of the pdp11, but it never did in any way resemble a macro assembler as far as I know.

And in fact, as gizmo86 pointed out, one of the explicit goals of C in terms of "making unix easier to write" was, afaik, that it would allow the code to be more portable to other architectures, because as the 11 in the name implies, the history that led to the creation of unix was not short on new architectures, and one of the key weaknesses of Multics was, afaik, the fact that it was rigidly written specifically for one specific architecture.

By this logic nearly every programming language is just a macro assembler at a certain level of (non-)complexity. All compilers do is translate syntax (macros?) into machine code (assembly?). So, is pascal also a macro assembler?

Anyways here is a list of instructions on the pdp11 that you can't directly represent even in the earliest versions of C to my knowledge:

  ROR
  ROL
  SWAB
  SXT
  ADC
  SBC
  BVC
  BVS
  BCS
  MARK
  HALT
  WAIT
  RESET
  MTPD
  MTPI
  MFPD
  MFPI
  MTPS
  MFPS
  MFPT
  CLC
  CLV
  CLZ
  CLN
  CCC
  SEC
  SEV
  SEZ
  SEN
  SCC

That's the distinguishing factor of a "compiler" vs. a "macro assembler". A macro assembler is a layer on top of an assembler that still allows you full access to the instruction set and allows control flow that most compilers would consider unsafe. C was never that. It wasn't derived from a language that was that and it didn't, to my knowledge, allow that even on pdp-11 unix.

I'd be really happy to see some code from one of the code archives of unix' history that proves me wrong there, though, if you have it.

UNIX was originally written in Assembly, C was used to rewrite the 4th version.

Unfortunately, classes teaching C often start with telling people that with C they learn how the real machine and the real CPU works on the lowest levels.

I'd be surprised if this was true. It certainly doesn't match my experience, and it also doesn't really make sense. The difference between machine code, assembly, and programming languages like C is first-week CS 101 stuff. What are you basing this on?

Have any good sources of this topic of macro assemblers? I'm interested in reading about this type of thing. I'll search Google, but you can't always find good sources there today.

Yeah, plenty of books at archive.org.

"Assembly language for the PC"

https://archive.org/details/assemblylanguage00soch

"TASM Reference"

https://archive.org/details/bitsavers_borlandturemblerVersio...

"Structured Assembler Language For IBM Microcomputers"

https://archive.org/details/structured-assembler-language-fo...

Or just go to MASM documentation, you will find plenty of directives that look like an high level language

https://docs.microsoft.com/en-us/cpp/assembler/masm/directiv...

Unfourtuantly not. The only one I have worked with was really just a normal assembler with a big library of in house macros.

I searched a bit, and found a large set of pages with information on the topic in the context of the 360 architecture. I'd say this architecture is unique in that it is still to this day used to write high level software.

Put "360 assembler macros" in your search engine and you'll find lots of information.

This is a good place to start: https://www.ibm.com/docs/en/zos/2.1.0?topic=language-macro-i...

Sorry to break it to you: https://m-cacm.acm.org/magazines/2018/7/229036-c-is-not-a-lo...

Even x86 assembly is not a low-level language: https://blog.erratasec.com/2015/03/x86-is-high-level-languag...

Not only x86 assembly, microcoded CPUs has always been quite common on CISC designs, hence why 1960's papers tend to refer to Assembly as bytecode.

I've read that and it would take a book to explain whats wrong with it.

> I do not expect it in any way to abstract away the CPU

C was especially designed to do that! https://stackoverflow.com/questions/53100198/what-is-the-pre...

Feel free to examine the resulting machine code. Godbolt is a really valuable tool.

"run as written" implies targeting an abstract machine that performs operations the way programmer imagines it does. This machine isn't specified anywhere.

The hard part is that programmers don't imagine the code to be executed in a naive unoptimized way, but still expect it to be reasonably well optimized (converted to "obviously" equivalent assembly), except where optimizations combine into a result they haven't thought of.

This is more of a multi threading & ordering topic than a compiler topic. There were never any guarantees about another thread discretely observing those two observable operations in the first place, due to the memory model.

Not to mention that a big part of the confusion is not even the compilation - it's not beyond possibilities to inspect the machine code - but the processor itself still does a lot of unintuitive stuff, even more so with concurrency. I wish the tooling you mention existed, but also some microcode simulation/implementation to look beyond that.

I’m not sure what are you thinking of, but CPUs are guaranteed to run as if the given instructions were executed one after another, even though in practice it is free to reorder them slightly.

Memory operations aren't required to behave like that, although on x86 they mostly do. ARM and Alpha certainly don't.

> CPUs are guaranteed to run as if the given instructions were executed one after another

That’s just not true. Intel models will reorder reads and other processors will even reorder writes - and I’m talking about observably.

I've dealt with many CPUs designed to run out of sequence and to guarantee that. I've dealt with multiprocessor computers where order is not guaranteed at all outside the use of hardware interlinks. I've dealt with embarrassingly parallel processors where instruction order is orthogonal.

Perhaps you're thinking of a simple abstract CPUs used in "introduction to computers" courses instead of the real-life silicon used in most applications?

> Well, think of it this way, if you're not into compilers wouldn't you expect your code to run as written?

But the possible and actual interleavings aren't written in the program - so how can I guess what the user needs from what's written?

Consider the reverse. What if the interleaving was undesirable, would you like some way to be informed that it’s impossible so you don’t have to reason about it?

When trying to reason about code, finding somethings that looks like a bug but isn’t can result in a lot of wasted effort.

> What if the interleaving was undesirable, would you like some way to be informed that it’s impossible

That's like synchronised in Java or any other kind of atomic block or serialising block - that's already a thing.

That information is moving in the wrong direction. I don’t want the programmer to be telling a compiler what to do, I want the compiler to inform the programmer what it did.

> That information is moving in the wrong direction.

You set the direction not me! You asked me 'would you like some way to be informed that [the interleaving is] impossible'. Yes... and the user gives me that information through a synchronised block. If they don't tell me they want an interleaving to be impossible then why should I not make it possible? There are countless other interleavings I make impossible or possible when I do any trivial compiler task such as scheduling. Should I consult the user on all of them? I don't think you'd like all the questions you'd get!

Informing the user isn’t about what you did, it’s about the user having some way of knowing what you did. At least compilers give you assembly code to look at, CPU’s are little better than black magic from a programmers perspective.

The user can dump the compiler output and intermediate structures if they want.

Or they could program against the spec and not worry.

Check the generated machine/byte code?

The answer is: don't guess.

We don't - and in this example we'd combine atomics rather than guess that the user wanted the thread to yield between them, since they never told us that.

That's not true, you are making a guess: that the user doesn't care about the intermediate stage. Not guessing would mean generating the code as specified. It doesn't matter if you think its a bad idea, the whole ethos of C is trust the programmer.

> you are making a guess: that the user doesn't care about the intermediate stage

That's not a guess - the user didn't put anything in between them so we know they haven't required them to be separate.

Keeping them separate would be making the guess - a guess that they need them to be separate. Why should we invent that constraint when the user never wrote it?

> Not guessing would mean generating the code as specified.

We are generating it as specified - nothing in the specification says atomic actions have to be run independently. Why should I invent extra constraints that don't exist in the specification?

Oh come on, that language lawyering is sophistry. If I write 4 instructions that are "atomic" I expect 4 instructions. If I end up with one, they werent really "atoms" were they? I do not expect what you just said. I may think, gee, perhaps I don't need 4 instructions, but what you are doing is undeniably changing the meaning of the program, even if you think it's close enough. The fact that there's an article about this tends to agree with me that most people think: wtf happened?

> If I write 4 instructions that are "atomic" I expect 4 instructions.

Why do you expect that? Nobody ever promised you that. If you expect something you weren’t promised then that’s on you.

> If I end up with one, they werent really "atoms" were they?

Atomic has a technical definition - it means either applied fully or not at all. That doesn’t preclude combining.

> what you are doing is undeniably changing the meaning of the program

It doesn’t! According to the spec we’re both working off. If you think the spec is wrong then change it. But I can’t guess what you want unless you tell me using the spec.

> Why do you expect that?

Because I wrote it?

> Nobody ever promised you that.

I just assumed that compiler writers had empathy for humans, my bad.

> It doesn’t! According to the spec we’re both working off. If you think the spec is wrong then change it. But I can’t guess what you want unless you tell me using the spec.

How big is the spec and how many working programmers do you think are super intimately familiar? Do you think this article would be on the front page if this information wasn't surprising? Are you building a compiler for humans, or are you just chasing metrics and getting the smug satisfaction of telling users "I told you so" when shit doesn't work in an intuitive way?

If you want to beat me over the head with "the spec says", you win, master language lawyer. I build things for a living and I like it when the code I specify is the code that's generated. You can blame users all you want, but maybe after the trillionth zero day exploit because nobody gets these things right you might consider empathy and developer ergonomics.

> I like it when the code I specify is the code that's generated

But you aren't specifying it!

You're asking me to follow extra constraints that you haven't written down and we haven't agreed. Don't you think it's unreasonable when you say it out loud like that?

If you want extra constraints on top of what is already agreed in the spec then we can do that, but you have to ask for it and you have to bring me a workable spec. You want me to 'intuit' but really you want me to read your mind and follow a lot of unwritten rules!

I think we have a fundamental difference on what a compiler should do. I want the compiler to take my exact instructions and generate the best machine code it can, that follows those exact instructions. You want to rewrite my program to be more optimal. The tension here is that, even though I'm sure you can do that, I have to debug the result, and the result I have to debug is now stripped of symbols and super hard to relate back to my original source code. So ok, I am only adding one constraint but it's a big one: don't ever change the line-by-line meaning of my source code. If I wrote a line of code I intended for it to be executed, I don't want my memsets removed or my atomics collapsed, not because you can't do it, but because I need to debug it. I don't think taking advantage of undefined behavior to create optimizations is useful because being wrong 10% faster isn't really that useful. And I think optimizations like removing memsets are a really bad idea; not only does it potentially make peoples data insecure but the only people that are ever going to find a bug like that are people looking for vulnerabilities.

And frankly, here's the thing: I don't care that much about most of your optimizations. If I'm making my code fast, what I'm fretting about is designing my data structures to fit in cache, or using SIMD instructions, or offloading work to the GPU or other coprocessors. Screwing with the meaning of my code just makes my life harder.

But the meaning of your program is not being changed or screwed with. The optimised version implements the same meaning as your original program.

The problem is you don't understand the meaning of what you've written. You think the language has extra meaning that it doesn't. You can already get the meaning you want through existing constructs like volatile and yield and memset_s.

> The problem is you don't understand the meaning of what you've written.

This is the problem with the attitude of compiler writers. Of course there are bugs. Nobody perfect. Pobodys nerfect? If I accidentally left a sign overflow in the code, or something of that nature, I think any (human) reading the code would understand what I meant, but the compiler is like "'Lets get weird!'". Yes I CAN get the meaning through the existing construct. That, however, is not my point. I am perfectly skilled and capable of doing this. What I'm questioning is why tool vendors think it's useful or practical to take advantage of users mistakes. I'd rather have a diagnostic of "why are you adding these numbers multiple times?" then have the compiler silently rewrite something I did on purpose.

I'm not writing this in the abstract, I'm writing this because things like this have hurt me in the past. It's super annoying. I don't care about your optimization to make things five percent faster if I spend three days debugging why the release build crashes if the window is full screen, but the debug build runs fine 100% of the time. The code has a bug, I own this, but I don't see how the attitude of "if your code has a bug, we will turn it into fucking dragons" is even remotely useful. Usually it's not even my code I'm fixing, it's someone elses. I just want people to stop breaking things to meet a dumb benchmark.

It sounds like you're railing against optimising based on undefined behaviour. That's a separate thing - that's not what's being discussed in this article or thread - this is all well-defined behaviour.

> I want the compiler to take my exact instructions and generate the best machine code it can, that follows those exact instructions.

Two problems.

1. Your definition of "exact" is not exact. It is exact in your mind, but it isn't going to be precisely the same across all developers and compiler authors cannot reasonably develop hard requirements from this. This is especially true when targeting different machines. Operations on a uin64_t aren't going to look the same on a 32 bit machine, for example.

2. Attempting to do this makes your programs slow as hell. You can claim that compiler optimizations don't matter... but you are free to deploy debug builds if you really think this. C and C++ especially aim to provide a sort of ambient performance where everything is fast rather than just providing speed for hot loops. People complain nearly daily about modern applications being slow and then turn around and say we should throw away decades of compiler optimizations.

I think you're severely underestimating how much slower everything would be if compilers weren't allowed to optimize anymore.

Here's where I start when I'm optimizing things:

- Does my data fit in cache

- Can I make it fit in cache?

- Have I packed my memory efficiently?

- Am I using the right algorithm?

"Can the compiler inline this" is like 5000 on my list of things I care about. Compiler optimizations are mostly only useful to people who werent paying attention in the first place.

> "Can the compiler inline this" is like 5000 on my list of things I care about. Compiler optimizations are mostly only useful to people who werent paying attention in the first place.

Okay, you're definitely severely underestimating how much slower everything would be if compilers weren't allowed to optimize anymore.

Yeah. For somebody who really cares a lot about cache misses he sure doesn't seem to be concerned about basic things like leaving virtual function calls in place.

> I build things for a living and I like it when the code I specify is the code that's generated.

Okay, then specify -O0 and enjoy your dumpster fire performance.

-O0 doesn't necessarily change this. After all, optimizations aren't allowed to violate the spec (if they do, they're buggy). So any code that can be output by an optimizing compiler could (in principle) be output by a non-optimizing compiler.

To be fair, it's usually like 5% slower. But I also don't usually have control over the flags if I'm writing a library.

I’d be curious to know what benchmarks you’re using on which O0 is only 5% slower. I suspect your program must be almost entirely IO-bound, if this is the case. In that case, Python, Ruby, or shell probably wouldn’t be much slower either.

Writing games, I've rarely seen the optimizer do anything of substantial use.

If you're going to use atomics, you should be reading what guarantees those atomics actually provide. In this case of memory_order_relaxed they pretty clearly specify they don't provide any cross-thread synchronization. So if you wrote those 4 atomics expecting something the docs explicitly & clearly say it doesn't provide, that's pretty much entirely your fault and not "language lawyering"

'atomic' isn't uncuttable in the sense that it is irreducible, but in the sense that it cannot be stopped/does not produce intermediaries. If you write four atomic instructions you should expect that they will not be interrupted, nothing more. of course I'm not an expert, but I think that is where your misunderstanding has originated.

> 'atomic' isn't uncuttable

Well then it's a pretty bad name no? Considering what the word atomic means.

Did you even read the post you're replying to. "Uncuttable" can apply to multiple properties of an instruction's execution - "atomic" colloquially refers to one of these properties.

You seem to want "atomic" to mean "the data that corresponds to the instruction cannot be divided" but it means "the execution of the instruction cannot be divided", those aren't the same thing.

The four hardest problems in computer science are cache invalidation, naming stuff, and off by one errors. Please accept that your understanding was flawed and have empathy for the people who named stuff.