News for hackers. Stuff that matters.
Re:Zig's Curious Multi-Sequence for Loops by brundolf on Monday February 27, 2023 @11:24AM (#34958280)
Rust also hates hidden allocations, and its iterator system can do all of this without them

Although- thinking about it, that may rely on the borrow checker (move semantics specifically)

Re:Zig's Curious Multi-Sequence for Loops by masklinn on Monday February 27, 2023 @11:43AM (#34958584)
That's nothing special though, `zip` just takes an item from each iterator, packs them into a tuple, and yields that. It has no weird bounds or requirements or anything: https://doc.rust-lang.org/std/iter/fn.zip.html

The impl of the default `next` is:

    fn next(&mut self) -> Option<(A::Item, B::Item)> {
        let x = self.a.next()?;
        let y = self.b.next()?;
        Some((x, y))
    }
So completely straightforward.
Re:Zig's Curious Multi-Sequence for Loops by defen on Monday February 27, 2023 @12:35PM (#34959362)
Can that zip more than two iterators? And does it perform a bounds check on each call to `a.next()` and `b.next()`?
Re:Zig's Curious Multi-Sequence for Loops by brundolf on Monday February 27, 2023 @12:38PM (#34959395)
It stops when one of the two iterators ends

It can't zip more than two per se, but you could zip the result of the first zip into a third and get ((item1, item2), item3). You could then map these if you wanted, to flatten them into a single tuple .map(|((item1, item2), item3)| (item1, item2, item3))

Of course there's a trade-off here between ergonomics and generality

Re:Zig's Curious Multi-Sequence for Loops by masklinn on Monday February 27, 2023 @12:43PM (#34959476)
> It can't zip more than two per se, but you could zip the result of the first zip into a third and get ((item1, item2), item3). You could then map these if you wanted, to flatten them into a single tuple .map(|((item1, item2), item3)| (item1, item2, item3))

FWIW that's more or less what `itertools::izip!` does for you, it just chains `zip`s then "splats" them using a `map`.

Re:Zig's Curious Multi-Sequence for Loops by defen on Monday February 27, 2023 @12:44PM (#34959493)
> It stops when one of the two iterators ends

Right; my question is, suppose you're iterating over two slice iterators - won't each call to `a.next()` and `b.next()` have to check whether that sub-iterator is done? One of the benefits of the Zig approach is that you can iterate over an arbitrary number of slices and do one check before entering the loop, followed by the compiler emitting unchecked index access in the loop. So it basically compiles down to the equivalent of a C `for` loop.

Re:Zig's Curious Multi-Sequence for Loops by masklinn on Monday February 27, 2023 @12:58PM (#34959682)
Rust's zip has a specialisation for iterators with a trusted length. Such as slice iterators.

`zip` yields exactly the same assembly as a loop over the index range with an unsafe item access: https://godbolt.org/z/7ebfxbhxc

Re:Zig's Curious Multi-Sequence for Loops by brundolf on Monday February 27, 2023 @01:32PM (#34960135)
Interesting, are "trusted-length" iterators something that might ever make it into userspace? Maybe as const generics?
Re:Zig's Curious Multi-Sequence for Loops by masklinn on Monday February 27, 2023 @01:39PM (#34960222)
It’s already in userspace, though nightly (and unsafe, obviously), so weather it’ll be stabilised, and in what form, is an open question: https://doc.rust-lang.org/std/iter/trait.TrustedLen.html
Re:Zig's Curious Multi-Sequence for Loops by kristoff_it on Monday February 27, 2023 @03:10PM (#34961256)
You have to pass -O though, the point of Zig's for loop syntax is to get fast compile times and good performance also in debug mode :^)
Re:Zig's Curious Multi-Sequence for Loops by defen on Monday February 27, 2023 @03:22PM (#34961389)
That's cool. At the same time though, it almost feels like a distinction without a difference in some ways - Zig has a special built-in syntax; Rust doesn't use special syntax, but it does use complex special-cased unsafe code in the stdlib in order to implement a safe + performant API.
Re:Zig's Curious Multi-Sequence for Loops by masklinn on Monday February 27, 2023 @03:29PM (#34961472)
On the other hand, the "special cased unsafe code" is applicable to more than just zip, more than just the one array type, and is available in userland (though currently unstable so nightly only, both to implement it on a bespoke type and to rely on it).
Re:Zig's Curious Multi-Sequence for Loops by brundolf on Monday February 27, 2023 @03:44PM (#34961605)
Rust's is built on top of (and exposed to) Iterators, which are a very general concept that can be rooted in all kinds of data structures, composed in all kinds of ways, and collected/processed in all kinds of ways (i.e. the user's code might not even contain an actual loop). The code continues to work in many situations, even where the optimization doesn't apply

You trade some special-case syntax and ergonomics for that generality, but it is very general even if not all of it is optimized in the same way

Re:Zig's Curious Multi-Sequence for Loops by the8472 on Monday February 27, 2023 @03:25PM (#34961426)
For Zip it's TrustedRandomAccess[0] instead of TrustedLen. Imo the most radioactively unsafe trait in the standard library and will likely never be stabilized in its current form.

[0] https://github.com/rust-lang/rust/blob/f540a25745e03cfe9eac7...

Re:Zig's Curious Multi-Sequence for Loops by chlorion on Monday February 27, 2023 @08:28PM (#34964555)
You can zip a zipper to combine them, but you end up with an item like (a, (b, (c, d))), not a huge deal since it gets destructured automatically.

The itertools crate has a multizip function/macro that allows zipping as many iterators as you'd like without nesting zips inside of zips!

I really recommend checking out itertools if you use rust and like iterators! It's a direct dependency of rustc itself too, which suggests its a trusted project (it may even be maintained by rust devs, not totally sure).

Re:Zig's Curious Multi-Sequence for Loops by brundolf on Monday February 27, 2023 @11:56AM (#34958789)
I was thinking about the fact that whatever you're iterating over has to be copied around throughout the process. Rust can guarantee that eg. deep-copies (clones) of allocated structs will never happen implicitly, if your iterator owns the values being iterated. But in languages where copying can trigger allocations, this could be a problem

I don't actually know whether that applies to Zig though

Re:Zig's Curious Multi-Sequence for Loops by laserbeam on Monday February 27, 2023 @11:00PM (#34965459)
You're assuming that an iterator gets created and returned by zip. In zig the for loop syntax doesn't need the concept of an iterator. All you need is variables in a capture.

That's what I mean by straightforward. You'd have to argue about the memory layout for an iterator, that you call functions on it etc. Lots of small decisions. In zig those things would usually be part of the standard library, not language features.

If you want to call a next() function on a thing in zig, you want to be explicit. That's what the language means by "no hidden control flow". You'd use a while loop if you wanted to iterate on a hashmap for example.

In some sense, this is less elegant, but it's usually more obvious what the compiler would end up doing in any bit of code.

Re:Zig's Curious Multi-Sequence for Loops by gpanders on Monday February 27, 2023 @01:30PM (#34960109)
>Rust also hates hidden allocations

Does it? Rust seems happy to allocate silently all the time.

let x = String(“hi”); let y = vec![];

Do either of these allocate? As the writer or reader of this code, how do I know if either of these statements result in a heap allocation, or if the data is strictly on the stack?

Zig’s requirement of explicitly passing around an Allocator type removes any ambiguity completely.

Re:Zig's Curious Multi-Sequence for Loops by brundolf on Monday February 27, 2023 @01:38PM (#34960207)
Sure, any arbitrary function (or macro) logic can allocate. It's more a philosophy, not something that's language-enforced in Rust- if you're creating a mutable, variable-size data structure like a String or a Vec or a HashMap you're not going to be very surprised that it allocates at some point (though technically zero-length Vecs don't allocate on construction, they wait until an item is added)

But closures don't require allocation, iterators don't require allocation, async doesn't require allocation. Copy semantics also don't allow allocation- implicit copies can only happen for data structures that are bitwise-copyable, which is enforced by the compiler. For copy-with-allocation you have to implement the Clone trait, and then invoke it explicitly with the .clone() method

But the original context was a question of philosophy, so I was only speaking to Rust's overall philosophy

Re:Zig's Curious Multi-Sequence for Loops by tialaramex on Monday February 27, 2023 @08:16PM (#34964459)
> Technically I think if you're using no_std you won't have access to any standard constructs that allocate

Indeed, because you only have core, and not alloc, which is where the allocator lives.

Mostly, core looks very similar to the functionality you can get of the same types in std - except that of course types which allocate are missing (Vec, HashMap, Box, String, etc.) and all the types which reflect operating system services are likewise missing (UdpSocket, File, Mutex, etc.)

However there are deviations. For example, Rust's slices in core don't have a sort() method. Why not? Rust provides a stable sort which uses an allocator because that's much faster, it doesn't bother providing a not-so-good stable sort that can work without using an allocator, if you can't afford an allocator but you want stable sort that's a rather niche case, Rust won't solve it. Rust's sort_unstable() doesn't need an allocator and so that is provided on slices even with only the core library.

Rust For Linux, the Linux kernel's Rust, has core, plus its own somewhat customised twist on alloc, plus an entirely custom kernel library. In Rust For Linux, the allocators are all explicit because Linus hates implied allocation. So for example in Rust For Linux a Vec doesn't have a push() method, because push() on a Vec can cause the Vec to grow, which is an allocation - instead Rust For Linux provides Vec::try_push() which fails if you'd need to grow the Vec before it could successfully push.

Re:Zig's Curious Multi-Sequence for Loops by steveklabnik on Monday February 27, 2023 @01:36PM (#34960170)
(You’re forgetting a “new” in the string example)
Re:Zig's Curious Multi-Sequence for Loops by gpanders on Monday February 27, 2023 @03:18PM (#34961349)
Thanks, I fixed it :)
Re:Zig's Curious Multi-Sequence for Loops by tialaramex on Monday February 27, 2023 @08:41PM (#34964620)
For what it's worth (I can't tell if you knew) vec![] doesn't allocate. Actually in this context it likely won't even compile, although it depends.

vec![] says I want a Vec with nothing in it. That doesn't require an allocation. In effect you are asking for three things, a pointer, of the correct type but not actually pointing at anything, and two integers (length and capacity) which are both zero.

Rust's first question here is: A vec of what? If we wrote something in those square brackets, it could guess the type of that, but we didn't. It may be able to infer the type from what is pushed into this Vec named y later, except we didn't make it mutable so we can't push anything into it - or if it's returned, from the return type (Rust does not allow function parameter or return types to be inferred). If Rust isn't able to infer the type, that's an error during compilation.

  let mut y: Vec<()> = vec![];
That says I want a mutable but initially empty Vec of the empty tuple. Rust is fine with that, and the result is a Vec which can hold up to isize::MAX of the empty tuple. It won't allocate, because the empty tuple doesn't take up any actual space - it's empty. Since empty tuples are indistinguishable in some sense Rust is really destroying them when you put them in the Vec and then making fresh ones to order when you remove them and there's just no way to tell.

  let mut y: Vec<u8> = vec![];
Now we're making a mutable, initially empty, Vec of bytes. This still doesn't allocate... yet. However if we try to push a byte into it, or we ask to ensure there is space for one or more bytes in it, that will allocate.

  let y: Vec<u8> = Vec::with_capacity(1);

This allocates immediately. Even though we promised we won't actually mutate this Vec, we insisted it be created with capacity for at least one byte, which will mean an allocation. Rust may end up allocating for a modest capacity larger than one byte, since it's likely the underlying system works in larger "chunks".
Re:Zig's Curious Multi-Sequence for Loops by veber-alex on Tuesday February 28, 2023 @02:53AM (#34966773)
> Zig’s requirement of explicitly passing around an Allocator type removes any ambiguity completely.

Not really...

    var list = std.ArrayList(u21).init(std.testing.allocator);
    try list.append('a');
Does creating the ArrayList allocate memory? Is appending to the array list reallocates? The only thing you know is that ArrayList will sometime use the allocator, you can't tell when. And this gets worse once the abstraction level increases, if you have your own type which accepts an allocator and has 100 methods it's impossible to tell which methods allocate.

In both languages you still need to know the specifics of the data structures you are working with, the only real problem that Zig solves is to actually allow you to use custom allocators with the std collections, which is something that is still unstable in Rust.

Re:Zig's Curious Multi-Sequence for Loops by tomck on Tuesday February 28, 2023 @01:07PM (#34972625)
You can tell when, because if it uses the allocator it will return an error. So the first line definitely doesn't allocate, and the second definitely does.

That is, unless you explicitly handle OOM conditions inside your construct, e.g. 'crash if you're OOM', which isn't typical in zig code. All code I interact with will return an allocator error if allocation fails.

Re:Zig's Curious Multi-Sequence for Loops by alpaca128 on Tuesday February 28, 2023 @05:40AM (#34967845)
The Rust docs are very clear that initializing an empty vector does not cause an allocation and is thus equivalent to using `Vec::with_capacity(0)`. The same applies to strings because strings are vectors of bytes with additional methods.

Whether a clearly visible `vec.push(x)` call is silent to you depends on your viewpoint. I'd say it's explicitly recognizeable as potential allocation, it has to be when pushing to a dynamic structure of arbitrary size - but I also understand how this can be seen differently. This is, I think, where Rust and Zig differ a lot. Rust makes most things explicit but still allows things like this or macros, Zig tries to avoid even that.