News for hackers. Stuff that matters.
Re:Rust – A hard decision pays off by throwaway894345 on Wednesday August 24, 2022 @12:17PM (#32581549)
I actually don't care as much for iterators as I thought I would. Beyond some simpler map().reduce() stuff they tend to fall over pretty fast, and I end up spending too much time trying to make iterator chains work before defaulting to for loops. I also dislike how anemic Rust's stdlib is. I'm sure there are good reasons, but I like that I can just reach for Go's stdlib for annotating errors or dealing with JSON. Other than that, I like the API design in Rust's stdlib.

Here's an example where the iterator-chain version is way more complex:

    let posts = read_dir(source_directory)?
        .filter_map(|result| {
            (|| {
                let entry = result?;
                let os_file_name = entry.file_name();
                let file_name = os_file_name.to_string_lossy();
                if Self::is_bundle(&entry)? {
                    self.parse_post_bundle(&file_name, &entry.path())
                        .map(Some)
                } else if file_name.ends_with(MARKDOWN_EXTENSION) {
                    self.parse_post(
                        file_name.trim_end_matches(MARKDOWN_EXTENSION),
                        &mut File::open(&entry.path())?,
                    )
                    .map(Some)
                } else {
                    Ok(None)
                }
            })()
            .transpose()
        })
        .collect::<Result<Vec<_>>>()?;
Here's the imperative version:

    let mut posts = Vec::new();
    for result in read_dir(source_directory)? {
        let entry = result?;
        let os_file_name = entry.file_name();
        let file_name = os_file_name.to_string_lossy();
        if Self::is_bundle(&entry)? {
            posts.push(self.parse_post_bundle(&file_name, &entry.path())?);
        } else if file_name.ends_with(MARKDOWN_EXTENSION) {
            posts.push(self.parse_post(
                file_name.trim_end_matches(MARKDOWN_EXTENSION),
                &mut File::open(&entry.path())?,
            )?);
        }
    }
Re:Rust – A hard decision pays off by 0x457 on Wednesday August 24, 2022 @04:56PM (#32585632)
Iterator-chain version would be a lot better looking if it was actually was a chain and not everything pushed into a single `filter_map`. "Ifs" are weird looking, one branch is using `entry`(I take it's a `DirEntry`?) and another is using `file_name` I suspect both could use the and look like `Self::is_` ?

If you're going to write an iterator chain in an imperative way, then you might as well right it imperatively.

A few notes:

- you can compare `&OsStr` with `&str` without `to_string_lossy` any explicit conversion (`&str -> &OsStr` done in `PartialEq` is basically free)

- you can get extension from `Path` by using `Path::extension`

- you can get filename without extension by using `Path::file_stem` or `Path::file_prefix`

- making your parse functions take the same arguments makes it a lot cleaner

Something like this:

    read_dir("")?
    .map(|e| e.map(|e| e.path()))
    .map(|path| {
        path.and_then(|path| {
            if Self::is_bundle(&path) {
                self.parse_post_bundle(&path.file_name().to_string_lossy(), &path).map(Some)
            } else if Self::is_markdown(&path) {
                self.parse_post(&path.file_stem().to_string_lossy(), &path).map(Some)

            } else {
                Ok(None)
            }
        })
    })
    .filter_map(Result::transpose)
    .collect::<Result<Vec<()>, std::io::Error>>()

    // Ignore this, just an example
    fn is_bundle(path: &Path) -> bool {
        path.extension().map(|ext| ext == "bundle").unwrap_or(false)
    }
    
    fn is_markdown(path: &Path) -> bool {
        path.extension().map(|ext| ext == "md").unwrap_or(false)
    }
Re:Rust – A hard decision pays off by Already__Taken on Wednesday August 24, 2022 @08:13PM (#32587641)
Forgive me for asking instead of looking, I'm still getting into rust; why do you map instead of getting the result?

    path.extension().map(|ext| ext == "bundle").unwrap_or(false)
vs

    (path.extension() == "bundle").unwrap_or(false)
or further up

    .map(|e| e.map(|e| e.path()))
    vs
    .map(|e| e.path())
Is the .map keeping any value wrapped in an iterator so you can map again? I thought it was interesting what you wrote looks a lot like what I'd write in powershell.
Re:Rust – A hard decision pays off by steveklabnik on Wednesday August 24, 2022 @09:47PM (#32588400)
Not your parent, but I do know the answer.

extension() returns an Option<&OsStr>, because not every file has an extension https://doc.rust-lang.org/stable/std/path/struct.Path.html#m...

map on option says "run this closure on the value if it's Some, or do nothing if it's None https://doc.rust-lang.org/stable/std/option/enum.Option.html...

So after the map, we now have an Option<bool>. The unwrap_or method will say "give me the value if it's Some, and give me the argument if it's not https://doc.rust-lang.org/stable/std/option/enum.Option.html...

That means we finally end up with a bool.

  (path.extension() == "bundle").unwrap_or(false)
This code wouldn't work because you're comparing an Option<&OsStr> to a &str. Even if that worked, you'd have a boolean, and unwrap_or doesn't make sense on booleans, you'd just leave off that part.
Re:Rust – A hard decision pays off by Already__Taken on Monday September 05, 2022 @05:40AM (#32722301)
Awesome thanks, I wasn't on a machine to play around with it at the time.
Re:Rust – A hard decision pays off by 0x457 on Saturday August 27, 2022 @08:39PM (#32624647)
`extension()` gives you an `Option`, remember `Path` might not be a file or file could be without extension.

Why not `((path.extension() == Some("bundle"))`? I don't know, it didn't work, and I didn't want to spend too much time on it.

`read_dir` gives us an iterator over `Result<DirEntry>`, so `e` in that map is a `Result` and not a `DirEntry`. It's two different "map" functions - on Option<T> and Result<T,E> it maps "T", but on Iterator it maps "Item" from that iterator.

Here is an explanation why `read_dir` give you a Result - https://doc.rust-lang.org/std/fs/fn.read_dir.html#errors

Re:Rust – A hard decision pays off by Already__Taken on Monday September 05, 2022 @05:39AM (#32722294)
Thanks for the insight
Re:Rust – A hard decision pays off by throwaway894345 on Wednesday August 24, 2022 @08:32PM (#32587788)
I'm sure there's plenty of room for improvement. I spent several hours with a bunch of Rustaceans just trying to get the short-circuiting to work right, and we sort of gave up on it beyond that.
Re:Rust – A hard decision pays off by mamcx on Wednesday August 24, 2022 @12:50PM (#32582183)
The thing is in Rust this is totally ok. Is NOT un-idiomatic to mix imperative + functional paradigms in the same block.
Re:Rust – A hard decision pays off by throwaway894345 on Wednesday August 24, 2022 @01:06PM (#32582448)
Agreed, but I think people make too big a deal about iterators. The simple for loops that they’re fit to replace aren’t that hard to read or write, so they aren’t causing real problems, and on the more complex end you’re writing imperative stuff anyway. Iterators certainly aren’t a bad thing, but they’re not the game changer that Go-critics made them out to be in all of the Rust-vs-Go debates.
Re:Rust – A hard decision pays off by mustache_kimono on Wednesday August 24, 2022 @03:33PM (#32584614)
> Agreed, but I think people make too big a deal about iterators.

I may just have more time to dick around, but I've been forcing myself to write everything as an iterator in Rust (instead of a for loop) just to learn the paradigm and, honestly, it's been pretty great.

First, there is a performance impact. You may think this would be small, but, in my experience, it can be relatively large. There must be several reasons of which I am ignorant, but they probably include mutable no alias inside closures, bounds check elision, etc.

Second, there is a mutability advantage to just using filter/flatten/map/collect for 90% of your use cases.

Third, the more advanced iterator methods will cover the other 5-10% of your use cases, and will make your code much easier to read, once you understand how to use them (if you're trying to learning something/anything more complex at the same time, save your brain power!). I don't have a single traditional for loop in my personal projects.

My take on iterators is, for iterator/Rust beginners (like me!/perhaps not you) -- the best way to learn is to get it working as imperative code, and when you have the time, go back a rewrite as an iterator. In the beginning, it's super frustrating. But you'll get better and better and, after awhile, you'll much, much prefer writing as an iterator. You won't even think of using a for loop.

Re:Rust – A hard decision pays off by throwaway894345 on Wednesday August 24, 2022 @04:58PM (#32585659)
I mean, I definitely have tried to rewrite stuff as iterators even when it isn't convenient. I only got to that working example by dedicating a few hours with the good people on the Rust Discord trying to make it work (a lot of seasoned Rustaceans also weren't able to get it working, it was a collaborative effort that took a fair amount of time and rhapsodizing). And even then, the result is pretty ugly (I don't think many seasoned Rustaceans would call it "idiomatic" or advise the first version over the second).
Re:Rust – A hard decision pays off by mustache_kimono on Wednesday August 24, 2022 @06:09PM (#32586445)
Agreed. Your example is a pretty pathological case, but yeah.
Re:Rust – A hard decision pays off by efnx on Wednesday August 24, 2022 @04:36PM (#32585392)
The selling point of iterators isn’t that they replace for loops, it’s that you can make new iterators by implementing a trait. That at way downstream users can use your iterators in a for loop. Of course you also can use a more functional style if you like.
Re:Rust – A hard decision pays off by nomel on Wednesday August 24, 2022 @01:34PM (#32582858)
I disagree. If I can avoid an index variable, I’m going to have less bugs since I won’t have to index.
Re:Rust – A hard decision pays off by throwaway894345 on Wednesday August 24, 2022 @03:25PM (#32584502)
I'm not sure what you mean. You don't have to deal with index variables in a for loop:

    for item in iterator {
        vector.push_back(foo(item));
    }
You're not explicitly indexing the source iterator or the destination vector at any point.
Re:Rust – A hard decision pays off by nomel on Wednesday August 24, 2022 @03:41PM (#32584713)
Agreed. That's why I enjoy iterators, and use them whenever possible. I think they're important.
Re:Rust – A hard decision pays off by throwaway894345 on Wednesday August 24, 2022 @04:11PM (#32585114)
But earlier you claimed the for-loop version required index variables and indexing into iterators/vectors/etc ... ?
Re:Rust – A hard decision pays off by nomel on Wednesday August 24, 2022 @07:33PM (#32587308)
I clearly said the opposite, in response to, what appeared to be, your suggestion that indexing isn't so bad:

"I think people make too big a deal about iterators. The simple for loops that they’re fit to replace aren’t that hard to read or write, so they aren’t causing real problem"

Re:Rust – A hard decision pays off by MauranKilom on Wednesday August 24, 2022 @08:13PM (#32587639)
It seems like you imply that iterators = range-for and "simple loops" = indexing, which is clearly not what the previous poster has in mind (which was "range-for = imperative loops, iterators = functional style, simple loops = trivial loop body").
Re:Rust – A hard decision pays off by throwaway894345 on Wednesday August 24, 2022 @08:37PM (#32587813)
Yes, I was responding to:

> I disagree. If I can avoid an index variable, I’m going to have less bugs since I won’t have to index

Which (given thread context) sounds pretty clearly like he's advocating against `for item in iterator {...}` and in favor of iterator chains. If there's another way of reading that comment, I'm not seeing it. :)

Re:Rust – A hard decision pays off by lr1970 on Wednesday August 24, 2022 @05:21PM (#32585942)
> If I can avoid an index variable, I’m going to have less bugs since I won’t have to index.

Not only that. In safe Rust vector indexes are always bound-checked that slows execution a little bit. Iterators over a vector, on the other hand, do not suffer from bound-check overhead.

Re:Rust – A hard decision pays off by lvass on Wednesday August 24, 2022 @01:43PM (#32582993)
The first version isn't what I'd write in a purely functional language either, a recursive version would be much saner. Isn't recursion idiomatic in rust?
Re:Rust – A hard decision pays off by steveklabnik on Wednesday August 24, 2022 @01:51PM (#32583139)
Rust doesn't guarantee TCO and so recursion is a bit dicer than in many functional languages.
Re:Rust – A hard decision pays off by throwaway894345 on Wednesday August 24, 2022 @03:27PM (#32584522)
In addition to Steve's sibling comment, I'm responding narrowly to iterators versus for-loops. Recursion is out of scope.
Re:Rust – A hard decision pays off by thinkharderdev on Wednesday August 24, 2022 @01:31PM (#32582810)
Yep, agree wholeheartedly with this. I came to Rust from Scala where writing an explicit for loop is looked at as one step above using goto :) and it took me a while to just give in and write the imperative version.
Re:Rust – A hard decision pays off by SoftTalker on Wednesday August 24, 2022 @12:22PM (#32581650)
I guess that's subjective, not knowing rust I see a few more lines in the first example but beyond that it does not jump out at me as being "way more complex."
Re:Rust – A hard decision pays off by throwaway894345 on Wednesday August 24, 2022 @12:45PM (#32582081)
Hah, it took me and a bunch of other people in the Rust discord a fair amount of time to figure out how to get the first example to compile correctly in the first place, and I find the “transpose()” business to be particularly convoluted. I think even Rust people would prefer the imperative version. I also suspect there’s some hindsight benefit at play in that it’s easier to make sense of the first snippet than it was to write it.
Re:Rust – A hard decision pays off by tialaramex on Wednesday August 24, 2022 @01:06PM (#32582461)
Yes, the imperative version looks like the sane choice to me. Rust's iterator documentation does suggest that this is not a language which favours iterator chains everywhere.

However one reason to prefer an iterator chain in some cases is that the chain might imply an optimisation you'd otherwise have to write manually and might not think of.

For example if I have N things, and I map them, perhaps more than once, but I'm definitely getting N of whatever the output of the last map was, the iterator chain might very well see that I don't actually have any way to change N and so when I collect() that into a Vec the Vec it gives me is constructed with capacity N, saving the (amortized but non-zero) cost of growing it during insertions.

Imperatively I can remember to write Vec::with_capacity(N), and that's safe of course but it's an extra step to remember.

The imperative approach does particularly shine on the opposite edge of this, if I know that I'm getting no more than 100 items out of this pipeline, despite putting N in, Rust almost certainly won't see that and won't make Vec::with_capacity(100) for a collect() call whereas my imperative code can just explicitly write Vec::with_capacity(100) or whatever.

Re:Rust – A hard decision pays off by sgeisenh on Wednesday August 24, 2022 @01:35PM (#32582872)
Depending on how you write it, iterators are much better at this since most iterator methods produce iterators with size hints.

If you use "take" for example, collect should do the right thing: https://doc.rust-lang.org/src/core/iter/adapters/take.rs.htm...

Re:Rust – A hard decision pays off by tialaramex on Wednesday August 24, 2022 @02:08PM (#32583425)
Mere hints aren't enough for collect()

It will only care about your hints if you implement the unsafe trait TrustedLen (which promises those hints are correct)

Take is indeed TrustedLen if the Iterator it's taking from is TrustedLen (as in this case it can be sure of the size hint)

If you get to ~100 via some means other than take()ing 100 of them, chances are you don't have TrustedLen as a result.

Re:Rust – A hard decision pays off by Alexendoo on Wednesday August 24, 2022 @02:55PM (#32584077)
TrustedLen will mean it can safely take the upper bound, but Vec for example will still use the lower bound of the hint when something isn't TrustedLen

https://doc.rust-lang.org/src/alloc/vec/spec_from_iter_neste...

Re:Rust – A hard decision pays off by tialaramex on Wednesday August 24, 2022 @03:14PM (#32584353)
Ooh nice, I hadn't seen that. OK, so there are probably places where I assumed the iterator won't do as nice a job of collect() as I did manually but I was wrong.

I wonder how the hell I didn't see that when I was tracking down how collect() ends up caring about TrustedLen.

Thanks.

Re:Rust – A hard decision pays off by ywei3410 on Wednesday August 24, 2022 @01:08PM (#32582471)
Honestly the closure + `transpose` is really ugly as sin. I had to do a double take and reason through the steps which you took to get that particular snippet (while not reading the imperative one).
Re:Rust – A hard decision pays off by throwaway894345 on Wednesday August 24, 2022 @03:28PM (#32584542)
Thanks, I feel validated. (:
Re:Rust – A hard decision pays off by nu11ptr on Wednesday August 24, 2022 @02:50PM (#32584018)
There is no need to force something iterator style. By all means use a loop if it is more readable. Also, if your iterator style closure body gets very verbose, you can always pull the logic out into a function or closure and pass it in. In the end, both styles are idiomatic, and the best choice is which makes it more readable (subjective of course).

I personally use iterator-style for 1-3 simple ops, but go into a loop for anything more complicated. The functional part of me wants to force iterator style but I fight it for complex use cases because it is irrational on my part, and I find loops easier to read for stuff like that.

Re:Rust – A hard decision pays off by lijogdfljk on Wednesday August 24, 2022 @04:15PM (#32585156)
Fair, but i'd probably write your iterator example differently. Though, besides the point perhaps - as i agree there are plenty of cases where imperative approaches are cleaner.

I however will often have a dozen maps/filters/etc and that style in imperative is what, nesting repeatedly or using `continue`. Meh.

If your Iterator consists of a single `filter_map` i can see why you don't get much value out of them in that case. It's because.. well, there's not much value to get out of it.

Re:Rust – A hard decision pays off by drran on Wednesday August 24, 2022 @01:27PM (#32582750)
Now, compare a parallel version of both.
Re:Rust – A hard decision pays off by throwaway894345 on Wednesday August 24, 2022 @03:32PM (#32584603)
It's not obvious to me that the parallel version of the first is going to be cleaner (or more performant) than a parallel version of the second, but even if it is, I don't agree with mangling (to put it nicely) single-threaded code on the off chance that it might be made parallel one day.