News for hackers. Stuff that matters.
Re:Linear Address Spaces: Unsafe at any speed by kazinator on Wednesday June 29, 2022 @04:49PM (#31925534)
> Why do we even have linear physical and virtual addresses in the first place, when pretty much everything today is object-oriented?

Simple: we don't want some low level kernel memory management dictating what constitutes an "object".

Everything isn't object-oriented. E.g. large arrays, memory-mapped files, including executables and libraries.

Linear memory sucks, but every other organization sucks more.

Segmented has been done; the benefit-to-clunk ratio was negligible.

Re:Linear Address Spaces: Unsafe at any speed by MaxBarraclough on Thursday June 30, 2022 @03:40AM (#31929723)
I'm reminded of an old comment that expressed a similar sentiment: we've discovered over time that OS Kernels aren't the place for Grand Abstractions.

[0] https://news.ycombinator.com/item?id=9807777

Re:Linear Address Spaces: Unsafe at any speed by 323 on Wednesday June 29, 2022 @07:02PM (#31926806)
Yet in practice the stack on x64 is separate from the heap, GPU memory is general is separate from system memory, and even inside the GPU you have at least a 3 level segmented memory hierarchy.
Re:Linear Address Spaces: Unsafe at any speed by gary_0 on Wednesday June 29, 2022 @07:40PM (#31927101)
Even in C-land, you're technically not supposed to fabricate arbitrary heap pointers, you're supposed to offset something you get from malloc(). The Standard says (if memory serves) that only pointers from the beginning of a malloc() to 1-past-the-end have defined behavior (when it comes to the heap).

Of course, there are probably lots of in-practice exceptions when it comes to embedded, kernel code, mmap() shenanigans, etc.

Re:Linear Address Spaces: Unsafe at any speed by e63f67dd-065b on Wednesday June 29, 2022 @09:11PM (#31927651)
Oh boy, welcome to the exciting world of C pointer provenance :) What you just described is what compiler people call pointer provenance, where each pointer has, in addition to the address, a second piece of info attached to it that describes all the places to which the pointer can point.

This is an extremely simplified and probably incorrect view of https://open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1434r0.... This is complicated because nobody agrees on what the correct behaviour should be, and we have mountains of legacy codebases that all rely on something slightly different when pointers get converted to ints and back.

Re:Linear Address Spaces: Unsafe at any speed by gary_0 on Wednesday June 29, 2022 @09:22PM (#31927722)
That's what makes C programming fun: doing what the compiler/language people say you shouldn't be doing and seeing if it works anyways. At least, it works on my machine...
Re:Linear Address Spaces: Unsafe at any speed by pjmlp on Thursday June 30, 2022 @01:47AM (#31929038)
Until the compiler gets upgraded....
Re:Linear Address Spaces: Unsafe at any speed by adrianN on Thursday June 30, 2022 @03:21AM (#31929592)
Until changes in seemingly far away code change the optimization decisions the compiler makes...
Re:Linear Address Spaces: Unsafe at any speed by phkamp on Thursday June 30, 2022 @03:51AM (#31929780)
That is actually the R1000's raison d'etre:

Rational built a truly semantic IDE, and the result of that is that if you change a line of source-code somewhere in a huge complicated project, it will know precisely what the consequences of that change are.

Ultimately, this allows it often just recompile that single line.

Re:Linear Address Spaces: Unsafe at any speed by verdagon on Wednesday June 29, 2022 @10:29PM (#31928158)
That sounds like the "generation" tag in Vale's generational references [0] or like how memory tagging works on ARM CPUs [1].

I love the approach, it's a way to get a lot more memory safety while not giving up a program's flexibility, especially in C's case.

Some languages are opting to disallow pointer arithmetic and conversion between integers and pointers. We'll see how it works out!

[0] https://verdagon.dev/blog/generational-references

[1] https://developer.arm.com/-/media/Arm%20Developer%20Communit...

Re:Linear Address Spaces: Unsafe at any speed by anonymoushn on Thursday June 30, 2022 @04:28AM (#31929993)
The part of this that I most enjoy is that you cannot even obtain some memory from your own allocator implemented on top of mmap or on top of malloc, store ints in it, return it to the allocator, ask the allocator for memory again and receive the same memory back, and store doubles in it.
Re:Linear Address Spaces: Unsafe at any speed by gpderetta on Thursday June 30, 2022 @05:28AM (#31930305)
Yes, notoriously you can't implement malloc in portable C.

In C++, I think they added enough magic in that you should now be able to do it (placement new, std::launder and numerous other hacks).

Re:Linear Address Spaces: Unsafe at any speed by rkangel on Thursday June 30, 2022 @11:15AM (#31933563)
I've implemented a pools allocator for embedded devices a couple of times. Allocating memory for callers out of a block of static memory. I thought it was portable C. Which detail of the standard did I not realise I was running into?
Re:Linear Address Spaces: Unsafe at any speed by gpderetta on Thursday June 30, 2022 @02:23PM (#31936457)
I assume the block of static memory was a large static char array.

My understanding that reusing the storage would violate the aliasing rules[1] and the rules against overlapping object lifetimes.

[1] while char ptrs can be used to access everything, the reverse is not allowed.

Re:Linear Address Spaces: Unsafe at any speed by teddyh on Thursday June 30, 2022 @09:29AM (#31932101)
Reportedly you can implement malloc in C, using mmap().
Re:Linear Address Spaces: Unsafe at any speed by gpderetta on Thursday June 30, 2022 @09:35AM (#31932165)
well mmap is posix, not standard portable C.
Re:Linear Address Spaces: Unsafe at any speed by raverbashing on Thursday June 30, 2022 @04:24AM (#31929976)
So instead of giving that metadata to the programmer the C language says F U and tells me only knowing where to put it is enough, right?

"hey you allocated it you should know about it right?" right yeah

Re:Linear Address Spaces: Unsafe at any speed by simiones on Thursday June 30, 2022 @09:09AM (#31931897)
To be clear, that metadata only (generally) exists in principle - it is not actually materialized and stored anywhere, even at compile time (definitely not at runtime). There may be cases where the compiler actually tracks it, but most often it is only tracked "best effort" - the compiler only needs to prove that two pointers can't be aliased based on any possible value of that metadata, not actually compute that metadata and use it.
Re:Linear Address Spaces: Unsafe at any speed by yuubi on Thursday June 30, 2022 @01:59AM (#31929091)
Standard C provides only a few ways to obtain valid pointers. implementations can define behaviors in cases that the standard leaves undefined, such as allowing more cases of casts of integers to pointers than the standard defines (common in embedded-land), or functions like mmap or sbrk. so you or your chip vendor could define FOO_REG as (uint32_t )0x80001234 and use it as if it were a variable.

Olde C used to just let you use integers as struct pointers, and there was only one struct member namespace. so code like this was valid and did an integer-size write to address 0177770. old unix did this for device register access; see the lions book.

struct { int integ; };

f() { 0177770->integ = 012345; }

Re:Linear Address Spaces: Unsafe at any speed by jasonhansel on Wednesday June 29, 2022 @09:33PM (#31927799)
Well, unless you're implementing malloc using e.g. sbrk().
Re:Linear Address Spaces: Unsafe at any speed by kllrnohj on Wednesday June 29, 2022 @10:22PM (#31928115)
Although you shouldn't be on a modern system, you'd instead be implementing malloc on top of mmap. So remove sbrk, and make mmap the "object allocator", and tada! You don't really need linear virtual address spaces anymore.
Re:Linear Address Spaces: Unsafe at any speed by gary_0 on Wednesday June 29, 2022 @10:56PM (#31928300)
Newer allocators like mimalloc[0] don't even support sbrk (I think jemalloc still does). Mimalloc seems to have some interesting features.

[0] https://github.com/microsoft/mimalloc

Re:Linear Address Spaces: Unsafe at any speed by hiker on Thursday June 30, 2022 @04:04AM (#31929864)
It still supports sbrk since it's available on WebAssembly but mmap is not.

https://webassembly.org/docs/faq/#what-about-mmap

Re:Linear Address Spaces: Unsafe at any speed by trasz on Thursday June 30, 2022 @09:53AM (#31932360)
Newer kernels like arm64 FreeBSD don't even support sbrk(2).
Re:Linear Address Spaces: Unsafe at any speed by astrange on Wednesday June 29, 2022 @11:28PM (#31928459)
Yes, this is why malloc() can't actually be implemented in C. Actual implementations of it exist because of special compiler dispensation, or mostly that the callers and implementations are in separate libraries so the implementation isn't visible to the caller.
Re:Linear Address Spaces: Unsafe at any speed by kazinator on Thursday June 30, 2022 @02:49PM (#31936818)
malloc perhaps can't be implemented in the abstract dialect called ISO C. It can be impemented in GNU C, and others.
Re:Linear Address Spaces: Unsafe at any speed by astrange on Thursday June 30, 2022 @07:51PM (#31940261)
It can't. GNU C adds a malloc attribute:

> Attribute malloc indicates that a function is malloc-like, i.e., that the pointer P returned by the function cannot alias any other pointer valid when the function returns, and moreover no pointers to valid objects occur in any storage addressed by P. In addition, the GCC predicts that a function with the attribute returns non-null in most cases.

But it doesn't provide any operations to do the things in this paragraph (create new pointers). The operation that does this is malloc itself.

This will typically not cause problems, but it would if LTO got so good you could include libc in it.

Re:Linear Address Spaces: Unsafe at any speed by kazinator on Thursday June 30, 2022 @04:28PM (#31938145)
So what? In a bitmask, such as a hardware register or database Bloom filter, two adjacent bits may be unrelated, and their relative position entirely coincidental and insignificant.

That doesn't mean we need a memory model in which bits are separate objects, in some sense.

Re:Linear Address Spaces: Unsafe at any speed by MarkSweep on Wednesday June 29, 2022 @06:11PM (#31926319)
The benefit-to-thunk ratio was not great either.

( one reference to thunks involving segmented memory: https://devblogs.microsoft.com/oldnewthing/20080207-00/?p=23... )

Re:Linear Address Spaces: Unsafe at any speed by kazinator on Wednesday June 29, 2022 @06:55PM (#31926711)
Real segmentation would have solved the problem described in the article. Under virtual memory segments like on the 80386 (and mainframes before that), you can physically relocate a segment and while adjusting its descriptor so that the addressing doesn't change.

The problem was mainly caused by having no MMU, so moving around objects in order to save space required adjusting pointers. Today, a copying garbage collector will do the same thing; rewrite all the links among the moved objects. You'd have similar hacks on Apple Macintoshes, with their MC68K processors and flat space, like doubly indirected objeccts managed by "handles" and whatnot.

Re:Linear Address Spaces: Unsafe at any speed by StillBored on Friday July 01, 2022 @01:26PM (#31949737)
The 386 segmentation system exists above the paging portion of the MMU, so it is possible to move them around without physically copying memory (or adjusting the segment+offset).

Here is a somewhat reasonable diagram of how it all works

https://stackoverflow.com/questions/4039325/assembly-segment...

(although i'm probably agreeing with you)

Re:Linear Address Spaces: Unsafe at any speed by Stratoscope on Wednesday June 29, 2022 @08:33PM (#31927436)
It's fun to see that old article again. (I wrote the FixDS program Raymond mentions at the end.)

One pain point in 16-bit Windows programming was that you had to do two special things for any callback function in your application: EXPORT the function in your .DEF file, and call MakeProcInstance() on your callback function to get an "instance thunk".

FixDS made a simple patch to the compiled function prologs that rendered all of this unnecessary. You could just use your callback function directly, like we expect today.

Re:Linear Address Spaces: Unsafe at any speed by bawolff on Thursday June 30, 2022 @12:10AM (#31928638)
Perhaps im misunderstanding, but it seems like you are using object-oriented in a different sense than the article does