News for hackers. Stuff that matters.
Re:Mistakes with Rust smart pointers: when Deref goes wrong by codeflo on Monday July 10, 2023 @02:13AM (#36662727)
I think I've hit the same kind of problem once. What I learned is that true smart pointers are types that (at least in Rust) aren't really supposed to have methods on their own, to avoid this type of ambiguity during method resolution.

For example, Box<T> implements Deref so that you can conveniently use T's methods. If you look at the documentation of Box, the things you can do with the Box itself, like Box::leak, are non-method functions. This means that you always have to qualify them, like Box::leak(my_box), and thus you can't get this type of conflict.

Using Deref for types like String or Vec doesn't really fall into this category. They implement something more analogous to an "is-a relationship" in OO. Even outside of those examples, I've seen people manually implement inheritance-like behavior, using Deref to delegate to a field that stores the "base class" instance.

If you do that, you're likely to have a name conflict between the two types sooner rather than later. That's fine as long as you're aware that Rust doesn't really do overload resolution -- AFAICT, the "outer" methods always wins regardless of the arguments at the call site.

Re:Mistakes with Rust smart pointers: when Deref goes wrong by planede on Monday July 10, 2023 @04:15AM (#36663436)
This feels analogous to hiding in C++. In C++ by default member functions in a base class don't overload with member functions of the same name in a derived class. They get hidden.
Re:Mistakes with Rust smart pointers: when Deref goes wrong by quietbritishjim on Monday July 10, 2023 @04:46AM (#36663614)
Not disagreeing with you but what's interesting is that C++ doesn't hide in this particular situation. Smart pointers implement operator-> and operator* but then ptr.foo() is still always the foo() method of the pointer type (and won't compile if it doesn't have it) - you need ptr->foo() or (*ptr).foo() to get the method of the pointed-to type.

It's one of the few places that C++ is more explicit than Rust. I wonder if there's a reason Rust went a different way? I'm mainly a C++ programmer so I'm genuinely curious.

Re:Mistakes with Rust smart pointers: when Deref goes wrong by OskarS on Monday July 10, 2023 @06:00AM (#36664123)
I've always liked that feature of C and C++. There's a lot of people that really hate it and in new languages they unify it to be just a dot. Zig does this as well, for instance. I always liked that the distinction was very explicit: use a.b() if you're calling b() on the object itself, use a->b() if you're de-referencing a first.

There are lots of things wrong with C and C++ that Zig and Rust fixes, but I never thought of this one being problematic in any way. It's as easy to type, the difference is obvious, and it's easy to see from a glance what's going on.

Re:Mistakes with Rust smart pointers: when Deref goes wrong by DixieDev on Monday July 10, 2023 @06:14AM (#36664213)
I'm mostly indifferent to it because it doesn't really harm readability, and it's not hard to know when to use it, but I can seem why it might be more strongly disliked. You can't entirely rely on it to know a dereference is happening because references (e.g. `int&`) aren't subject to the `->` requirement. It's also annoying if you find that you can refactor `func(T*)` to `func(T&)` and now you have to replace all `->`s with `.`s.
Re:Mistakes with Rust smart pointers: when Deref goes wrong by codeflo on Monday July 10, 2023 @06:18AM (#36664240)
> You can't entirely rely on it to know a dereference is happening because references (e.g. `int&`) aren't subject to the `->` requirement.

To the extent that this is a problem, it's a problem with references as a language feature in general, not with the arrow syntax.

Re:Mistakes with Rust smart pointers: when Deref goes wrong by quietbritishjim on Monday July 10, 2023 @11:11AM (#36667585)
> You can't entirely rely on it to know a dereference is happening because references (e.g. `int&`) aren't subject to the `->` requirement.

That's true but I think the bigger motivation is avoiding ambiguity than seeing when an indirection is happening. (You also can't see whether a method is virtual, i.e. indirecting via the vtable, from the call site.) In C++ references don't have any standalone methods or operators so there's no ambiguity from using methods via a reference just using a dot.

Re:Mistakes with Rust smart pointers: when Deref goes wrong by codeflo on Monday July 10, 2023 @06:14AM (#36664219)
I'm among the people who would have liked to eliminate even this special case and have a convenient postfix derefence for this. So instead of a->b() you'd have a*.b(), and if you need to dereference again, write a**.b() and so on. But as you said, lots of people really seem to hate this kind of thing. Zig is almost there with its dereference syntax, but still kept the magic dereferencing dot.
Re:Mistakes with Rust smart pointers: when Deref goes wrong by mananaysiempre on Monday July 10, 2023 @07:54AM (#36665074)
Object Pascal strikes again: if a: ^record ... end is a pointer to a record, then a^ is the pointed-to record as a value and a^.b is the way you access a field of it.

Incidentally, the authors of Go were asked why they didn’t use a postfix dereference when Sethi noted[1] back in 1981 that C syntax would have been simpler that way, was quoted in Ritchie’s 1993 retrospective[2] on that point, and even their own blog post on declaration syntax pointed out that possibility[3], but IIRC the answer amounted to “but then we wouldn’t have gotten those sweet, sweet C-bandwagon popularity points”.

(I also cannot help calling out “Language X” here: One nice approach that nevertheless basically forces implicit dereferencing is that of Algon-68, where all values are constant values; the way to declare a mutable variable is to declare a pointer—a constant value—to a mutable cell; and the way to make the whole thing bearable is to have the compiler infer the minimum possible amount of dereferencing in an expression from the types of the values involved. Then of course it becomes impossible to infer the types, so the ML family goes with the inconvenient dereferencing, and you end up not really using a lot of mutability in ML.)

[1] https://dx.doi.org/10.1002/spe.4380110606

[2] https://dx.doi.org/10.1145/154766.155580

[3] https://go.dev/blog/declaration-syntax

Re:Mistakes with Rust smart pointers: when Deref goes wrong by mr_00ff00 on Monday July 10, 2023 @11:10AM (#36667569)
I have limited Zig experience, but actually zig is different from rust or go in this regard, correct?

Rust and Go have the pointer have the unified syntax, but I thought in zig it does differentiate between the pointer itself and the value it points to.

Rust and Go:

ptr.method_on_ptr() ptr.method_from_underlying_val()

Zig:

ptr.method_on_ptr() ptr.*.method_from_underlying_val()

Re:Mistakes with Rust smart pointers: when Deref goes wrong by gpderetta on Monday July 10, 2023 @06:08AM (#36664175)
People have been trying to get 'operator.' overloading in C++ for the last 20 years. Unfortunately this confusion between pointer operations and pointee operations has always been an obstacle :(.
Re:Mistakes with Rust smart pointers: when Deref goes wrong by mostlylurks on Monday July 10, 2023 @06:42AM (#36664435)
One of the downsides of the C++ approach is that it makes it more cumbersome to make changes in your code. If you for instance make a trivial change where you change a variable from a stack-allocated object accessed directly via .foo() into a heap-allocated one accessed via ->foo(), you have to either manually change all .foo()s into ->foo()s or do some sort of weird work-around, such as making a separate variable that's a reference to the heap allocated object so that you can keep using those .foo()s. This is all relatively pointless busywork, which also pollutes the commit history with relatively meaningless changes.
Re:Mistakes with Rust smart pointers: when Deref goes wrong by lionkor on Monday July 10, 2023 @09:49AM (#36666406)
those changes in access are semantically different, so i prefer being forced to change code when semantics change
Re:Mistakes with Rust smart pointers: when Deref goes wrong by Guvante on Monday July 10, 2023 @11:22AM (#36667760)
There is no sematic difference between passing in a reference and passing in a copy of a smart pointers though.

Yes there is a difference at the call site between taking a reference to an owned thing and taking potential ownership of the thing.

However inside the function block it is noise. Nothing inside the code is semantically different between the two forms, both do work on preexisting data.

Certainly the second version could have a change where it does something with respect to the fact it is a smart pointers, changing what used to be a grabbed pointer to a weak pointer to be more explicit about lifetimes for instance.

However that code change would be just as explicit as long as there aren't any methods on the smart pointer in the auto deref world.

Re:Mistakes with Rust smart pointers: when Deref goes wrong by qalmakka on Monday July 10, 2023 @12:52PM (#36669081)
Deref for Vec<T> and String is fine because they are owning containers that decay to their "view" types (slices). Strings and vectors own memory, while views borrow it and can be used to operate on them without reallocating or moving the container itself, treating it as a contiguous list of values. A similar relationship exists between C++'s `std::string` and `std::string_view`, and `std::vector` and `std::span`.

IMHO there the logic behind that is very clear and makes a lot of sense. I think it's natural that view-related traits (the ones who operate on the elements of a contiguous area of memory) should be associated with slices, while memory-related ones (the ones that operate on an owning container) should be implemented on the owning types. If you mix them up, you are bound to mess up first and foremost because _it is not logically sound_. You don't need to own a string in order to check if all of its ASCII characters are all caps, for instance, because it's an operation that's also valid on any array of bytes, without the whole concept of "owning memory".

Re:Mistakes with Rust smart pointers: when Deref goes wrong by edbaskerville on Monday July 10, 2023 @03:11PM (#36671445)
Thank you article, parent, GP. You've saved me just in time from never having read the Deref documentation properly. I have some code to delete before I make this mistake again.
Re:Mistakes with Rust smart pointers: when Deref goes wrong by tialaramex on Monday July 10, 2023 @03:18PM (#36671545)
> You don't need to own a string in order to check if all of its ASCII characters are all caps, for instance, because it's an operation that's also valid on any array of bytes, without the whole concept of "owning memory".

Interestingly you can ask whether a string is ASCII, "this".is_ascii() is a predicate which does exactly that, but for what I think is your question, "are they all ASCII caps?" you'd need to go via an iterator and a lambda: "THIS".bytes().all(|b| b.is_ascii_uppercase())

The iterator is because there is no provision of each such predicate over the whole string, which seems reasonable (the character class predicates are provided on u8 (ASCII only), and on char (all of them)), but the lambda is due to Rust's 1.0 compatibility commitment plus an unfortunate historical choice. If not for compatibility we'd fix it so you could just write "THIS".bytes().all(u8::is_ascii_uppercase)

Re:Mistakes with Rust smart pointers: when Deref goes wrong by saghm on Monday July 10, 2023 @08:27PM (#36674986)
Am I correct in thinking this is the type of thing change that fits the criteria to be implemented in a new edition? Assuming the issue is the same one I often run into (i.e. 'a versus for<'a>), it seems like special casing this for closures passed as arguments would be quite nice, but the fact that it wasn't done for 2018 or 2021 makes me think that maybe it's not something that would qualify for introducing in this way.
Re:Mistakes with Rust smart pointers: when Deref goes wrong by tialaramex on Tuesday July 11, 2023 @04:06AM (#36677859)
It's not "just" syntax, so it couldn't "just" be a new edition, but I suppose it's conceivable that a hack could be invented and then the old behaviour grandfathered into prior editions as was done for impl IntoIterator for [T; N] in 2021 Edition.

That's a heavy lift though, you may need to carry this special code around in every compiler, forever, and I haven't even figured out what such code should look like, it might be really hard to even write it.

Re:Mistakes with Rust smart pointers: when Deref goes wrong by saghm on Tuesday July 11, 2023 @12:31PM (#36683395)
Good to know! I appreciate the insight
Re:Mistakes with Rust smart pointers: when Deref goes wrong by 3836293648 on Tuesday July 11, 2023 @12:41PM (#36683541)
How would removing the lambda affect back compat?
Re:Mistakes with Rust smart pointers: when Deref goes wrong by marcosdumay on Monday July 10, 2023 @03:29PM (#36671705)
> AFAICT, the "outer" methods always wins regardless of the arguments at the call site

I imagine this is explicitly state in some portion that I have never read of the Deref documentation. The thing is, every introductory text ignores this, and there is no itemized rule in a more global document so that things like Rc and Cell can point directly to it.

The Rust docs have a very general discoverability problem, and this is one of its facets. Since things have a very generic and reusable definition, the docs have no good place to put information like that.

Re:Mistakes with Rust smart pointers: when Deref goes wrong by saghm on Monday July 10, 2023 @08:48PM (#36675117)
It definitely would make sens to include this in the docs for `Deref`. I suspect the reason people hadn't thought to do that yet is because there's nothing specific to `Deref` about this behavior; it works the same whenever methods are present on a type due to trait definition[0] (although I imagine it's potentially less confusing for other traits given that the methods they provide are explicitly listed rather than implicit via the associated type).

It's also worth noting that if the ambiguity is between two trait impls rather than the concrete type and one of its trait impls, the compiler does require explicit disambiguation (either by calling the method as a top-level function on the trait and passing in the `self` parameter or by casting before calling the method)[1].

[0]: https://play.rust-lang.org/?version=stable&mode=debug&editio... [1]: https://play.rust-lang.org/?version=stable&mode=debug&editio...