News for hackers. Stuff that matters.
Re:Pointers Are Complicated, or What’s in a Byte? by iforgotpassword on Wednesday July 25, 2018 @02:40AM (#17607293)
I know what the post is trying to say and all the examples are fine and learning C(++) you should be aware of this and everything, but I think the conclusion of saying that this shows pointers aren't integers is an odd choice. They still are in memory. It's all about the interpretation of these integers, and the shortcuts compilers can take because the standard allows them to do so, as the author points out. If you turn off all optimizations or go ahead and write a really simple compiler yourself, you probably end up getting exactly the results you'd "expect" from those examples, at least on any platform that is not totally exotic. Also uninitialized bytes are still ordinary bytes, you just get a free rand() run. :)

Yeah maybe it's just me being weird.

Re:Pointers Are Complicated, or What’s in a Byte? by GolDDranks on Wednesday July 25, 2018 @03:08AM (#17607400)
Unitialized bytes can NOT be considered "ordinary bytes" on x86. This is because of memory paging and copy-on-write. If a memory is untouched, it is allowed to map to a different physical memory on different accesses, so the contents are not only "unknown", they may change under your feet! You must touch the memory page for this not to happen.
Re:Pointers Are Complicated, or What’s in a Byte? by gmueckl on Wednesday July 25, 2018 @05:08AM (#17607868)
This cannot possibly be correct. A read access to a region without a mapped page in the page table will cause the CPU to trigger a page fault exception. The OS must then decide whether to map memory there a one time operation or abort the program. A second read from the same page must lead to the same mapping then. There is no sane way for an operating system to switch out program heap pages after they have been mapped.
Re:Pointers Are Complicated, or What’s in a Byte? by ralfjung on Wednesday July 25, 2018 @07:02AM (#17608321)
This is looking at the wrong level of abstraction though. The compiler will already optimize your code in a way that multiple uses of the same uninitialized value can produce different results.

Arguing about these assembly/CPU-level details (unfortunately?) is besides the point when debating the semantics of languages like C, C++ or Rust.

Re:Pointers Are Complicated, or What’s in a Byte? by gmueckl on Wednesday July 25, 2018 @07:32AM (#17608440)
Grandparent already narrowed discussion down to x86 behaviour and I responded to details on that. This is definitely all outside the scope of language specifications.
Re:Pointers Are Complicated, or What’s in a Byte? by vardump on Wednesday July 25, 2018 @05:20AM (#17607924)
I read "memory is untouched" as no read/write accesses.

Because it'd be pretty odd for the page to change after first read from operating system implementation point of view. Generally I'd expect the default mapping to be to an all-zero physical memory page. (Unless there's a rowhammer attack (or similar) that successfully modifies said all-zero page...)

From C/C++ point of view, it's legal for uninitialized memory to arbitrarily change between (undefined behavior) reads until it is written to.

While I doubt any actual OS would modify the page in this manner after first read access, a C/C++ compiler can still return arbitrary (even random) values when reading from uninitialized memory. It doesn't even need to actually read from the memory proven to be uninitialized.

Re:Pointers Are Complicated, or What’s in a Byte? by gpderetta on Wednesday July 25, 2018 @06:34AM (#17608217)
madvise MADV_DONTNEED
Re:Pointers Are Complicated, or What’s in a Byte? by GolDDranks on Wednesday July 25, 2018 @07:11AM (#17608367)
I'm not going to take the responsibility whether it's correct or not - here's the talk I learned it from: https://youtu.be/kPR8h4-qZdk?t=22m17s
Re:Pointers Are Complicated, or What’s in a Byte? by gmueckl on Wednesday July 25, 2018 @07:57AM (#17608569)
I haven't watched all of it, but the explanation for the null terminator bug must contain some OS interaction telling the kernel that it is actually safe to discard the page containing the null terminator (e.g. the madvice call mentioned in a sibling post). If you're doing things like that, you're asking for trouble.
Re:Pointers Are Complicated, or What’s in a Byte? by _pmf_ on Wednesday July 25, 2018 @05:54AM (#17608071)
> There is no sane way for an operating system to switch out program heap pages after they have been mapped.

They'll even move the page to the harddisk if necessary. Or am I misunderstanding what you're asking?

Re:Pointers Are Complicated, or What’s in a Byte? by gmueckl on Wednesday July 25, 2018 @07:28AM (#17608430)
The OS can move the content of the pages in physical memory, but it will not make the page table entry point to a page with different content after that point.
Re:Pointers Are Complicated, or What’s in a Byte? by iforgotpassword on Wednesday July 25, 2018 @03:14AM (#17607423)
TIL, nice detail

(Not that I would've recommended using uninitialized bytes as an RNG anyways ;))

Re:Pointers Are Complicated, or What’s in a Byte? by vardump on Wednesday July 25, 2018 @02:50AM (#17607325)
Pointers are not integers in abstract model C++ compilers use for reasoning about code.

This is all about the abstract model used for optimization according to the language specification.

Re:Pointers Are Complicated, or What’s in a Byte? by iforgotpassword on Wednesday July 25, 2018 @03:11AM (#17607410)
Sure, I thought I made that clear in my post. Also I didn't want to imply that this is good code or code anyone should write. However I still think when learning C or explaining the language it is vastly helpful to go with the analogy of pointers being integers, since it is so much easier to grasp. Which doesn't mean that shouldn't come with a big warning not to write weird unsafe code and everything; but even if the standard would actually mandate that any form if crazy pointer arithmetic would be defined behavior I wouldn't exactly encourage anyone to write code like that.
Re:Pointers Are Complicated, or What’s in a Byte? by ralfjung on Wednesday July 25, 2018 @06:52AM (#17608288)
The sad truth is that I have seen, several times, people justifying real code doing questionable things with pointers in C(-like languages) by saying "well but pointers are just integers".

I am not sure what the best way to teach C is. Maybe it is a good idea to start with a "lie" and explain pointers as integers. But most C books/tutorials never go to the part where they explain that this is a lie, and that is a problem. The problem will only get worse as compilers get smarter and hence better at exploiting the UB that lurks in so many programs due to a naive treatment of pointers.

Re:Pointers Are Complicated, or What’s in a Byte? by IshKebab on Wednesday July 25, 2018 @03:39AM (#17607516)
There should be a different word for that though. A pointer is literally a stored memory address, i.e. an integer.

The thing the compiler manipulates could be called a "C pointer" or something.

Re:Pointers Are Complicated, or What’s in a Byte? by vardump on Wednesday July 25, 2018 @03:58AM (#17607595)
> A pointer is literally a stored memory address, i.e. an integer.

Consider:

  int a = 5;
  int* b = &a;
What happens when 'a' is allocated to a CPU register? If 'b' never escapes and no (undefined) pointer arithmetic is used, the compiler can simply consider '*b' as an alias to 'a'.

In this case the 'pointer' never actually points to memory.

> The thing the compiler manipulates could be called a "C pointer" or something.

Perhaps, although when talking about pointers, it's almost always in context of a particular programming language.

Maybe it'd be better to talk about "hardware pointers". Although on hardware level, from CPU point of view, a value or a register generally only becomes a "pointer" when it's used to compute a memory address for an instruction that accesses memory. (Excluding special hardware registers, like instruction pointer, page tables, DMA hardware or interrupt/exception vectors.)

So a "pointer" is always an abstraction in context of general purpose processing.

Re:Pointers Are Complicated, or What’s in a Byte? by klmr on Wednesday July 25, 2018 @07:05AM (#17608333)
> A pointer is literally a stored memory address, i.e. an integer

You’re mixing implementation with definition. A pointer is (usually) implemented in terms of an integer (i.e. its physical representation is a sequence of bytes interpreted as an integer offset in memory). But that isn’t what a pointer is. The C++ standard (or the C standard, or Rust, or any of many other languages) defines pointers as a concept that has all kinds of properties that integers don’t have. The whole point of this blog post is that this distinction is crucial in order to guarantee type safety and allow the compiler to perform basic optimisations.

Re:Pointers Are Complicated, or What’s in a Byte? by adwn on Wednesday July 25, 2018 @06:23AM (#17608167)
> A pointer is literally a stored memory address, i.e. an integer.

This is not correct in general. In x86 real mode, addresses consist of a segment and an offset. The linear address is computed as segment * 16 + offset. This is very different from a single integer, and even somewhat different from a pair of integers: Two different segment/offset-pairs can refer to the same linear address, so a pointer comparison has to behave differently than an integer-pair comparison.

Re:Pointers Are Complicated, or What’s in a Byte? by raverbashing on Wednesday July 25, 2018 @06:51AM (#17608284)
You're not wrong, but the segment/offset model is such a farce it's not even worth using as an example

(also you would have had to distinguish between far and near pointers)

At the processor level, pointers are distinguished either by the type of register or by the type of operation used on them (when they're more orthogonal, still there are "pointer only" registers like stack, ip, etc)