News for hackers. Stuff that matters.
Re:The memory models that underlie programming languages (2016) by littlestymaar on Thursday June 10, 2021 @03:39AM (#27457992)
Is it really different from C though? (But with some runtime intervention instead of UB)

Rc and Arc are maintaining the “be sure that you'll only free this once” invariant automatically. In C you have the same invariant, you just have have to maintain it by yourself (or use an ad hoc reference counting mechanism) and if you don't it's a double free (UB). And in both cases you are responsible ensure the memory is freed at least once.

Mutex ensure you don't perform one read and one write at the same time. This is also forbidden by the C memory model and you'll probably end up using a mutex too (or other concurrency primitives) with the exact same risk of deadlock.

Re:The memory models that underlie programming languages (2016) by moonchild on Thursday June 10, 2021 @04:31AM (#27458360)
C allows much more flexibility wrt lifetimes than does rust.

For example: a common pattern in video games is the frame-local allocation. You (cheaply) allocate some object at the beginning of a frame. Share references to it all over the place. At the end of the frame, you (cheaply) free all the objects that were allocated in this manner. This cannot really be expressed in terms of rust's ownership system.

More generally, rust enforces local correctness, while c allows for global correctness (a weaker condition). Consider, as a simpler example, this c function which I write frequently:

  char *tmpstr(void) {
   static char buf[16][1024];
   static int i = 0;
   char *ret = buf[i];
   i = (i+1) % 16;
   return ret;
  }
This function is not locally correct; in a vacuum, it is impossible to say whether this function violates invariants. (Ignoring the problem of overflowing the 1024-byte buffer.) There are many programs using this function which are correct; then, there are some programs which call this function more than 16 times in a row and end up with undesirable behaviour as a result.

Rust requires that every function be locally correct. Papering over some details: for any sound function f and hole-containing program P[_], if P[f] is well-typed then P[f] is sound; that means that it's always meaningful to consider f without considering the context in which it will be called. There is no such guarantee in c.

As a sound language, this is a very useful attribute for rust to have, because it means that libraries can be meaningfully type-checked (not to mention it significantly simplifies the type system), but it would be naive to say that rust simply formalizes the ownership model you would have used anyway in c.

Re:The memory models that underlie programming languages (2016) by drran on Thursday June 10, 2021 @07:15AM (#27459348)
You can use your own allocator in Rust, with your own rules, OR you can use static arrays and then just reset aray length to zero at the end of frame.

Your function is easy to write in (unsafe!) Rust:

https://play.rust-lang.org/?version=stable&mode=debug&editio...

  use lazy_static::lazy_static; // 1.4.0
  use std::mem;

  lazy_static! {
    static ref BUF: [String; 16] = [
      String::with_capacity(1024), String::with_capacity(1024), String::with_capacity(1024), String::with_capacity(1024),
      String::with_capacity(1024), String::with_capacity(1024), String::with_capacity(1024), String::with_capacity(1024),
      String::with_capacity(1024), String::with_capacity(1024), String::with_capacity(1024), String::with_capacity(1024),
      String::with_capacity(1024), String::with_capacity(1024), String::with_capacity(1024), String::with_capacity(1024)
    ];
    static ref BUF_INDEX: usize = 0;
  }

  fn tmpstr() -> &'static mut String { 
   let index_ptr: * const usize = &*BUF_INDEX;
   let index: &'static mut usize = unsafe { mem::transmute(index_ptr) };

   let buf_ptr: * const String = &BUF[*index];
   let ret: &'static mut String = unsafe { mem::transmute(buf_ptr) };

   ret.truncate(0);

   *index = (*index + 1usize) % BUF.len();
   
   return ret;
  }

  fn main(){
    let s1 = tmpstr();
    s1.push_str("Hello, world!");

    let s2 = tmpstr();
    s2.push_str("And hello world again!");

    dbg!(s1, s2);
  }
BTW: It's very good idea for embedded systems.
Re:The memory models that underlie programming languages (2016) by moonchild on Thursday June 10, 2021 @05:49PM (#27466260)
Yes, of course you can express this in rust. The topic at hand is whether it can be expressed in terms of rust's ownership policies. Your example shows that it cannot; it returns an object of static lifetime, avoiding the issue completely.
Re:The memory models that underlie programming languages (2016) by littlestymaar on Friday June 11, 2021 @03:22AM (#27470282)
The topic at hand was about the comparison between C and Rust in terms of memory model. Ownership policies are one restriction safe Rust uses to ensure that the user is adhering to its memory model, but Rust is not limited to safe Rust and you can op-out ownership policies if you use raw pointers.

The difference between Rust and C in terms of memory model isn't ownership, it's “mutable xor shared”.

Re:The memory models that underlie programming languages (2016) by drran on Friday June 11, 2021 @05:39AM (#27471107)
I cannot understand your question about "ownership policies". Rust have only one ownership policy: one owner per object.

It's possible to use pointers to circumvent the restriction and then control when object is destructed using ManuallyDrop[0], so burden of memory bookkeeping will be shifted from Rust compiler to the programmer.

[0]: https://doc.rust-lang.org/core/mem/struct.ManuallyDrop.html

Re:The memory models that underlie programming languages (2016) by scoutt on Friday June 11, 2021 @03:30AM (#27470331)
But String is heap allocated, right?

Can you just declare a static array and return a pointer pointing to any part of it? Like in the C example?

BUF is also visible to other parts of the program, correct? If so, it's not quite similar to the C example.

Re:The memory models that underlie programming languages (2016) by littlestymaar on Friday June 11, 2021 @05:03AM (#27470896)
> Can you just declare a static array and return a pointer pointing to any part of it? Like in the C example?

Indeed:

    lazy_static! {
      static ref BUF: [[u8;1024]; 16] = [
        [0;1024], [0;1024], [0;1024], [0;1024],
        [0;1024], [0;1024], [0;1024], [0;1024],
        [0;1024], [0;1024], [0;1024], [0;1024],
        [0;1024], [0;1024], [0;1024], [0;1024]];
      static ref BUF_INDEX: usize = 0;
    }
(You'll need to re-implement a `push_str` method on arrays yourself though)

> BUF is also visible to other parts of the program, correct? If so, it's not quite similar to the C example.

No, because it's not marked as `pub`, nobody outside the current module can access it.

Re:The memory models that underlie programming languages (2016) by drran on Friday June 11, 2021 @05:25AM (#27471029)
String is heap allocated by default, but String can be made from raw pointer to buffer and buffer length: https://doc.rust-lang.org/std/string/struct.String.html#meth... .

Yes, of course. BTW: You can use a c2rust converter to convert a C code to it exact equivalent in Rust.

No, buf is not exported, so it's "static" in C terminology.

Re:The memory models that underlie programming languages (2016) by scoutt on Friday June 11, 2021 @05:51AM (#27471197)
> No, buf is not exported, so it's "static" in C terminology.

Yes, I know that is not exported. What I wanted to say is that in the C example, no one outside tmpstr(void) can access static char buf[16][1024].

In the Rust example I can access static ref BUF from fn main() function, for example.

Re:The memory models that underlie programming languages (2016) by littlestymaar on Friday June 11, 2021 @07:55AM (#27472087)
> In the Rust example I can access static ref BUF from fn main() function, for example.

Only because in that simple example, BUF was defined directly in the main module.

In a more realistic exemple, BUF and tmpstr would belong to their own module, with only tmpstr being exported (and then accessible). See: https://play.rust-lang.org/?version=stable&mode=debug&editio...

Re:The memory models that underlie programming languages (2016) by masklinn on Thursday June 10, 2021 @06:15AM (#27458995)
> C allows much more flexibility wrt lifetimes than does rust.

Because C does not have lifetimes at all, or ownership as a formal langage concept, and thus does not run head first into the completeness problem.

Which is one of unsafe’s use cases: when the compiler can’t prove the code is correct (because there will always be correct programs the compiler can’t prove, and there are currently a fair bit of them), and you can, and there are no suitable safe alternatives, you use unsafe.

> For example: a common pattern in video games is the frame-local allocation. You (cheaply) allocate some object at the beginning of a frame. Share references to it all over the place. At the end of the frame, you (cheaply) free all the objects that were allocated in this manner. This cannot really be expressed in terms of rust's ownership system.

Of course it can, though it requires APIs which are not there (in the stdlib, yet): a frame local allocator and custom per-object allocator. All frame-local objects use that and thus keep a reference to the allocator.

And it avoids storing frame-local objects in locations which outlive the frame.

> This function is not locally correct; in a vacuum, it is impossible to say whether this function violates invariants.

That is also a use-case for unsafe: an unsafe function is one for which the caller is responsible for checking invariants.

This is unsafe anyway because modifying globals is unsafe to start with.

Re:The memory models that underlie programming languages (2016) by moonchild on Thursday June 10, 2021 @05:57PM (#27466337)
>> cannot really be expressed in terms of rust's ownership system

> Of course it can

You cannot share objects using rust's ownership system.

Interior mutability involves an unacceptable performance hit (and doesn't actually allow you to share anything). And if you use shared indices into a centralized array, then you have blinded the ownership system, creating pointers that it cannot make sense of.

>> This function is not locally correct; in a vacuum, it is impossible to say whether this function violates invariants.

> That is also a use-case for unsafe: an unsafe function is one for which the caller is responsible for checking invariants.

That's exactly right, and was my thesis; rust's type system does not allow you to prove the correctness of such a function, because it is neither correct nor incorrect without context. 'Unsafe', as a subversion of the type system which allows unsound behaviour, is completely uninteresting wrt the topic at hand; the comparison is a purely theoretical one between non-'unsafe' rust—that is, rust's ownership model—and c.

Re:The memory models that underlie programming languages (2016) by steveklabnik on Thursday June 10, 2021 @06:27PM (#27466647)
Interior mutability does not have uniform performance characteristics, Cell has a very different cost (that is, none) over RefCell (which involves checking a flag when accessing). Etc etc.

Unsafe does not allow unsound behavior, or rather, it lets you write unsound behavior but that is a bug. You still have to uphold various invariants in unsafe code. It’s there for when you know you can, but the compiler cannot, which is an inherent property of any sound system.

Re:The memory models that underlie programming languages (2016) by moonchild on Thursday June 10, 2021 @06:56PM (#27467018)
> Unsafe does not allow unsound behavior, or rather, it lets you write unsound behavior but that is a bug. You still have to uphold various invariants in unsafe code. It’s there for when you know you can, but the compiler cannot, which is an inherent property of any sound system.

I don't understand what you're saying here.

Rust-without-'unsafe' is a sound language; i.e. any program which types will be unable to violate runtime invariants.

Rust-with-'unsafe' is an unsound language; i.e. there are programs which type but yet violate runtime invariants.

That is a bug in those programs—sure, yes, of course. It is not a bug in rust; it is the language functioning as intended. However the topic at hand is the relationship between rust's type system and c's; hence unsafe is completely uninteresting, because it is a tool for subverting rust's type system.

(It occurs to me now that I might have gotten a less generally defensive response had I made the original comparison between rust and a hypothetical rust without unsafe. Oh well)

Re:The memory models that underlie programming languages (2016) by steveklabnik on Thursday June 10, 2021 @07:00PM (#27467049)
> However the topic at hand is the relationship between rust's type system and c's; hence unsafe is completely uninteresting,

Gotcha, I was just trying to clarify the semantics of unsafe, lots of people think it suddenly lets you do whatever you want, and while it will let you, that doesn't mean that you can, which is an important distinction. You're right that this puts it in the same general category of C in a theoretical perspective, I think practice will bear out that it's not a binary proposition, but time will tell :)

Re:The memory models that underlie programming languages (2016) by infogulch on Thursday June 10, 2021 @11:13PM (#27468885)
If manually managing memory is like wielding a gun, the borrow checker is an automatic safety that prevents you from pulling the trigger when you're roughly pointing it at yourself. But it's coarse-grained and errs on the side of caution; it simulates your foot as as the rectangular box that would contain it, not as a detailed 3D mesh. If you really think you can aim it between your toes and avoid hitting yourself (for example, "the value returned by this function must remain alive for no more than 15 successive invocations of this function"), unsafe will let you try, but the borrow checker's built-in rules isn't granular enough to help you, though it will still stop you if you accidentally put your hand in front without declaring it.

C is like a laser gun in a room full of oddly-shaped mirrors with no safety at all. Good luck.

Re:The memory models that underlie programming languages (2016) by moonchild on Friday June 11, 2021 @02:52AM (#27470126)
> it's coarse-grained and errs on the side of caution

That's what type systems in general do. There is no type system for a turing-complete language that is both sound and complete. (Trivial proof: if (undecidable) halt; unsound.)

Re:The memory models that underlie programming languages (2016) by littlestymaar on Friday June 11, 2021 @03:29AM (#27470324)
> That's exactly right, and was my thesis; rust's type system does not allow you to prove the correctness of such a function, because it is neither correct nor incorrect without context. 'Unsafe', as a subversion of the type system which allows unsound behavior, is completely uninteresting wrt the topic at hand;

For the record, the “topic at hand” was the memory model not the type-system, nor the theorem proving capabilities of the borrow checker.

Your arguments are not wrong per se, but they are completely missing the topic.

Re:The memory models that underlie programming languages (2016) by moonchild on Thursday June 10, 2021 @09:20PM (#27468245)
Upon further reflection, I don't think 'local/global correctness' distinction this was expressed very well or very usefully. The 'hole' example papers over so many details as to be misleading.

So here's another, hopefully better expression:

Rust's type system wrt lifetimes is not very expressive, compared with 'dynamically-typed' c. It can't represent things like 'the value returned by this function must remain alive for no more than 15 successive invocations of this function'.

Re:The memory models that underlie programming languages (2016) by flohofwoe on Thursday June 10, 2021 @04:09AM (#27458212)
A lot of useful C programs can be written without dynamic memory allocation or multithreading though, while Rust seems to assume a sort of "base complexity" even for very simple programs as soon as memory access is involved.
Re:The memory models that underlie programming languages (2016) by drran on Thursday June 10, 2021 @07:42AM (#27459523)
300 byte Blink for Arduino: https://github.com/vlisivka/rust-arduino-blink

Primitive pin tester for Arduino in 1kb: https://github.com/vlisivka/rapt

Re:The memory models that underlie programming languages (2016) by littlestymaar on Thursday June 10, 2021 @11:51AM (#27462114)
Not at all. (And that's actually the big reason why Rust is making its way to the Linux Kernel)
Re:The memory models that underlie programming languages (2016) by tialaramex on Thursday June 10, 2021 @10:52AM (#27461429)
Rust doesn't assume an allocator. The Rust standard library (to have things like growable strings) requires an allocator but the language and its core do not.

And Rust doesn't assume threading either. The default stuff isn't even thread safe, it's just that in Rust when you introduce threading and still try to use stuff that isn't thread safe the compiler will tell you that can't work.

You can deliberately write stuff that won't do what you meant of course, but they're trying to make it harder to do that by accident. I have a set of Rust types which defy logical behaviour (for example type whose members always compare as equal yet hash differently) and as intended if you use those in otherwise safe code your program might not work but it doesn't cause random craziness. For example maybe your attempt to binary search an ordered list of my data types that claim they form a total order and yet always reply "I'm smaller" when asked how they compare to anything, even themselves - will become an infinite loop, but it won't suddenly execute environment variables as code or anything like that whereas in C all bets are off.

Re:The memory models that underlie programming languages (2016) by littlestymaar on Thursday June 10, 2021 @11:48AM (#27462078)
> And Rust doesn't assume threading either. The default stuff isn't even thread safe

Right, and Rust doesn't even know what a thread is actually. It's all implemented in the standard library. The only thing Rust know, is that there exist some special Traits (Send and Sync) which are automatically implemented if a struct field implements it itself. And that's it.