I actually don't care as much for iterators as I thought I would. Beyond some simpler map().reduce() stuff they tend to fall over pretty fast, and I end up spending too much time trying to make iterator chains work before defaulting to for loops. I also dislike how anemic Rust's stdlib is. I'm sure there are good reasons, but I like that I can just reach for Go's stdlib for annotating errors or dealing with JSON. Other than that, I like the API design in Rust's stdlib.

Here's an example where the iterator-chain version is way more complex:

    let posts = read_dir(source_directory)?
        .filter_map(|result| {
            (|| {
                let entry = result?;
                let os_file_name = entry.file_name();
                let file_name = os_file_name.to_string_lossy();
                if Self::is_bundle(&entry)? {
                    self.parse_post_bundle(&file_name, &entry.path())
                        .map(Some)
                } else if file_name.ends_with(MARKDOWN_EXTENSION) {
                    self.parse_post(
                        file_name.trim_end_matches(MARKDOWN_EXTENSION),
                        &mut File::open(&entry.path())?,
                    )
                    .map(Some)
                } else {
                    Ok(None)
                }
            })()
            .transpose()
        })
        .collect::<Result<Vec<_>>>()?;

Here's the imperative version:

    let mut posts = Vec::new();
    for result in read_dir(source_directory)? {
        let entry = result?;
        let os_file_name = entry.file_name();
        let file_name = os_file_name.to_string_lossy();
        if Self::is_bundle(&entry)? {
            posts.push(self.parse_post_bundle(&file_name, &entry.path())?);
        } else if file_name.ends_with(MARKDOWN_EXTENSION) {
            posts.push(self.parse_post(
                file_name.trim_end_matches(MARKDOWN_EXTENSION),
                &mut File::open(&entry.path())?,
            )?);
        }
    }

Iterator-chain version would be a lot better looking if it was actually was a chain and not everything pushed into a single `filter_map`. "Ifs" are weird looking, one branch is using `entry`(I take it's a `DirEntry`?) and another is using `file_name` I suspect both could use the and look like `Self::is_` ?

If you're going to write an iterator chain in an imperative way, then you might as well right it imperatively.

A few notes:

- you can compare `&OsStr` with `&str` without `to_string_lossy` any explicit conversion (`&str -> &OsStr` done in `PartialEq` is basically free)

- you can get extension from `Path` by using `Path::extension`

- you can get filename without extension by using `Path::file_stem` or `Path::file_prefix`

- making your parse functions take the same arguments makes it a lot cleaner

Something like this:

    read_dir("")?
    .map(|e| e.map(|e| e.path()))
    .map(|path| {
        path.and_then(|path| {
            if Self::is_bundle(&path) {
                self.parse_post_bundle(&path.file_name().to_string_lossy(), &path).map(Some)
            } else if Self::is_markdown(&path) {
                self.parse_post(&path.file_stem().to_string_lossy(), &path).map(Some)

            } else {
                Ok(None)
            }
        })
    })
    .filter_map(Result::transpose)
    .collect::<Result<Vec<()>, std::io::Error>>()

    // Ignore this, just an example
    fn is_bundle(path: &Path) -> bool {
        path.extension().map(|ext| ext == "bundle").unwrap_or(false)
    }
    
    fn is_markdown(path: &Path) -> bool {
        path.extension().map(|ext| ext == "md").unwrap_or(false)
    }

Forgive me for asking instead of looking, I'm still getting into rust; why do you map instead of getting the result?

    path.extension().map(|ext| ext == "bundle").unwrap_or(false)

vs

    (path.extension() == "bundle").unwrap_or(false)

or further up

    .map(|e| e.map(|e| e.path()))
    vs
    .map(|e| e.path())

Is the .map keeping any value wrapped in an iterator so you can map again? I thought it was interesting what you wrote looks a lot like what I'd write in powershell.

Not your parent, but I do know the answer.

extension() returns an Option<&OsStr>, because not every file has an extension https://doc.rust-lang.org/stable/std/path/struct.Path.html#m...

map on option says "run this closure on the value if it's Some, or do nothing if it's None https://doc.rust-lang.org/stable/std/option/enum.Option.html...

So after the map, we now have an Option<bool>. The unwrap_or method will say "give me the value if it's Some, and give me the argument if it's not https://doc.rust-lang.org/stable/std/option/enum.Option.html...

That means we finally end up with a bool.

  (path.extension() == "bundle").unwrap_or(false)

This code wouldn't work because you're comparing an Option<&OsStr> to a &str. Even if that worked, you'd have a boolean, and unwrap_or doesn't make sense on booleans, you'd just leave off that part.

Awesome thanks, I wasn't on a machine to play around with it at the time.

`extension()` gives you an `Option`, remember `Path` might not be a file or file could be without extension.

Why not `((path.extension() == Some("bundle"))`? I don't know, it didn't work, and I didn't want to spend too much time on it.

`read_dir` gives us an iterator over `Result<DirEntry>`, so `e` in that map is a `Result` and not a `DirEntry`. It's two different "map" functions - on Option<T> and Result<T,E> it maps "T", but on Iterator it maps "Item" from that iterator.

Here is an explanation why `read_dir` give you a Result - https://doc.rust-lang.org/std/fs/fn.read_dir.html#errors

Thanks for the insight

I'm sure there's plenty of room for improvement. I spent several hours with a bunch of Rustaceans just trying to get the short-circuiting to work right, and we sort of gave up on it beyond that.

The thing is in Rust this is totally ok. Is NOT un-idiomatic to mix imperative + functional paradigms in the same block.

Agreed, but I think people make too big a deal about iterators. The simple for loops that they’re fit to replace aren’t that hard to read or write, so they aren’t causing real problems, and on the more complex end you’re writing imperative stuff anyway. Iterators certainly aren’t a bad thing, but they’re not the game changer that Go-critics made them out to be in all of the Rust-vs-Go debates.

> Agreed, but I think people make too big a deal about iterators.

I may just have more time to dick around, but I've been forcing myself to write everything as an iterator in Rust (instead of a for loop) just to learn the paradigm and, honestly, it's been pretty great.

First, there is a performance impact. You may think this would be small, but, in my experience, it can be relatively large. There must be several reasons of which I am ignorant, but they probably include mutable no alias inside closures, bounds check elision, etc.

Second, there is a mutability advantage to just using filter/flatten/map/collect for 90% of your use cases.

Third, the more advanced iterator methods will cover the other 5-10% of your use cases, and will make your code much easier to read, once you understand how to use them (if you're trying to learning something/anything more complex at the same time, save your brain power!). I don't have a single traditional for loop in my personal projects.

My take on iterators is, for iterator/Rust beginners (like me!/perhaps not you) -- the best way to learn is to get it working as imperative code, and when you have the time, go back a rewrite as an iterator. In the beginning, it's super frustrating. But you'll get better and better and, after awhile, you'll much, much prefer writing as an iterator. You won't even think of using a for loop.

I mean, I definitely have tried to rewrite stuff as iterators even when it isn't convenient. I only got to that working example by dedicating a few hours with the good people on the Rust Discord trying to make it work (a lot of seasoned Rustaceans also weren't able to get it working, it was a collaborative effort that took a fair amount of time and rhapsodizing). And even then, the result is pretty ugly (I don't think many seasoned Rustaceans would call it "idiomatic" or advise the first version over the second).

Agreed. Your example is a pretty pathological case, but yeah.

The selling point of iterators isn’t that they replace for loops, it’s that you can make new iterators by implementing a trait. That at way downstream users can use your iterators in a for loop. Of course you also can use a more functional style if you like.

I disagree. If I can avoid an index variable, I’m going to have less bugs since I won’t have to index.

I'm not sure what you mean. You don't have to deal with index variables in a for loop:

    for item in iterator {
        vector.push_back(foo(item));
    }

You're not explicitly indexing the source iterator or the destination vector at any point.

Agreed. That's why I enjoy iterators, and use them whenever possible. I think they're important.

But earlier you claimed the for-loop version required index variables and indexing into iterators/vectors/etc ... ?

I clearly said the opposite, in response to, what appeared to be, your suggestion that indexing isn't so bad:

"I think people make too big a deal about iterators. The simple for loops that they’re fit to replace aren’t that hard to read or write, so they aren’t causing real problem"

It seems like you imply that iterators = range-for and "simple loops" = indexing, which is clearly not what the previous poster has in mind (which was "range-for = imperative loops, iterators = functional style, simple loops = trivial loop body").

Yes, I was responding to:

> I disagree. If I can avoid an index variable, I’m going to have less bugs since I won’t have to index

Which (given thread context) sounds pretty clearly like he's advocating against `for item in iterator {...}` and in favor of iterator chains. If there's another way of reading that comment, I'm not seeing it. :)

> If I can avoid an index variable, I’m going to have less bugs since I won’t have to index.

Not only that. In safe Rust vector indexes are always bound-checked that slows execution a little bit. Iterators over a vector, on the other hand, do not suffer from bound-check overhead.

The first version isn't what I'd write in a purely functional language either, a recursive version would be much saner. Isn't recursion idiomatic in rust?

Rust doesn't guarantee TCO and so recursion is a bit dicer than in many functional languages.

In addition to Steve's sibling comment, I'm responding narrowly to iterators versus for-loops. Recursion is out of scope.

Yep, agree wholeheartedly with this. I came to Rust from Scala where writing an explicit for loop is looked at as one step above using goto :) and it took me a while to just give in and write the imperative version.

I guess that's subjective, not knowing rust I see a few more lines in the first example but beyond that it does not jump out at me as being "way more complex."

Hah, it took me and a bunch of other people in the Rust discord a fair amount of time to figure out how to get the first example to compile correctly in the first place, and I find the “transpose()” business to be particularly convoluted. I think even Rust people would prefer the imperative version. I also suspect there’s some hindsight benefit at play in that it’s easier to make sense of the first snippet than it was to write it.

Yes, the imperative version looks like the sane choice to me. Rust's iterator documentation does suggest that this is not a language which favours iterator chains everywhere.

However one reason to prefer an iterator chain in some cases is that the chain might imply an optimisation you'd otherwise have to write manually and might not think of.

For example if I have N things, and I map them, perhaps more than once, but I'm definitely getting N of whatever the output of the last map was, the iterator chain might very well see that I don't actually have any way to change N and so when I collect() that into a Vec the Vec it gives me is constructed with capacity N, saving the (amortized but non-zero) cost of growing it during insertions.

Imperatively I can remember to write Vec::with_capacity(N), and that's safe of course but it's an extra step to remember.

The imperative approach does particularly shine on the opposite edge of this, if I know that I'm getting no more than 100 items out of this pipeline, despite putting N in, Rust almost certainly won't see that and won't make Vec::with_capacity(100) for a collect() call whereas my imperative code can just explicitly write Vec::with_capacity(100) or whatever.

Depending on how you write it, iterators are much better at this since most iterator methods produce iterators with size hints.

If you use "take" for example, collect should do the right thing: https://doc.rust-lang.org/src/core/iter/adapters/take.rs.htm...

Mere hints aren't enough for collect()

It will only care about your hints if you implement the unsafe trait TrustedLen (which promises those hints are correct)

Take is indeed TrustedLen if the Iterator it's taking from is TrustedLen (as in this case it can be sure of the size hint)

If you get to ~100 via some means other than take()ing 100 of them, chances are you don't have TrustedLen as a result.

TrustedLen will mean it can safely take the upper bound, but Vec for example will still use the lower bound of the hint when something isn't TrustedLen

https://doc.rust-lang.org/src/alloc/vec/spec_from_iter_neste...

Ooh nice, I hadn't seen that. OK, so there are probably places where I assumed the iterator won't do as nice a job of collect() as I did manually but I was wrong.

I wonder how the hell I didn't see that when I was tracking down how collect() ends up caring about TrustedLen.

Thanks.

Honestly the closure + `transpose` is really ugly as sin. I had to do a double take and reason through the steps which you took to get that particular snippet (while not reading the imperative one).

Thanks, I feel validated. (:

There is no need to force something iterator style. By all means use a loop if it is more readable. Also, if your iterator style closure body gets very verbose, you can always pull the logic out into a function or closure and pass it in. In the end, both styles are idiomatic, and the best choice is which makes it more readable (subjective of course).

I personally use iterator-style for 1-3 simple ops, but go into a loop for anything more complicated. The functional part of me wants to force iterator style but I fight it for complex use cases because it is irrational on my part, and I find loops easier to read for stuff like that.

Fair, but i'd probably write your iterator example differently. Though, besides the point perhaps - as i agree there are plenty of cases where imperative approaches are cleaner.

I however will often have a dozen maps/filters/etc and that style in imperative is what, nesting repeatedly or using `continue`. Meh.

If your Iterator consists of a single `filter_map` i can see why you don't get much value out of them in that case. It's because.. well, there's not much value to get out of it.

Now, compare a parallel version of both.

It's not obvious to me that the parallel version of the first is going to be cleaner (or more performant) than a parallel version of the second, but even if it is, I don't agree with mangling (to put it nicely) single-threaded code on the off chance that it might be made parallel one day.

> but man Rust was a breath of fresh air with the amount of tooling that helped me solve problems

The tooling is the #1 reason I'd like to learn rust. I have not kept up with C++ and I'm not sure I ever will... sometimes plain C seems more straightforward. But those languages (C++ especially) have suffered from lack of standard tooling, in my opinion.

Even C# being a few years younger than Java seems to have made a huge difference in tooling availability (I'm sure having a single corporate driver during its lifetime made a difference, too).

I'm excited by the prospect of a compiled language with modern tooling.

Tooling was the reason I left C++. Having to do your own package management and script your own build system was deeply dull work. The IDE story was also pretty miserable, although I think clangd has improved it a bit. Getting decent information out of core dumps also sucked a lot. The language actually didn’t bother me too much after c++11, but everything else was such a bear that I jumped ship.

Rust has been very rewarding in that it largely fixes all of C++’s tooling and language problems. It very much is a significantly improved C++.

I actually filed a ticket at Carbon project saying c++'s top 1 problem is tooling, modern c++ is pretty nice to use already, no need for Carbon there in fact, at least not as needed as tooling.

Rust std lib is subpar compared to the Go one, yes you have iterators but that's it, basic things like async are not even provided just the interface so everyone has to use tokyo. Then for real use cases you're missing http/json/compression/crypto etc ...

https://pkg.go.dev/std

Overall tooling and std lib are better on Go, actually there are not many languages that are on part with Go to that regard.

When you see what you can do with the single go command, it's pretty powerful.

Agreed 100%.

Go doesn't have a standard library problem, it has a userland language problem.

* No sum types / algebraic data types in 2022.

* No exhaustive pattern matching in 2022

* No move semantics / Uses GC

* No borrow checker

* Still suffers from nil problem / No type-safe nil / No type-safe Optionals in 2022 -- (I can show you all the nil panics in kubernetes logs if you like)

Current go users are already sold so they don't expect more, and that's fine, but further evangelism will require a better host language.

Seems a bit odd to criticize Go for using GC rather than a borrow checker to manage memory. There’s surely a place for GCed languages, even if they’re not the right choice for every domain.

I don’t mind GC. I think it’s the right choice for the sort of applications Go targets. I wish it had algebraic data types (with exhaustive pattern matching), but the “in 2022” meme is just vapid snark.

Notably, despite its shortcomings, Go is still perhaps the most productive programming language for general purpose applications. It turns out that type system features are neat, but tooling, ecosystem, simplicity, etc outrank type system features by quite a lot. Rust does very well here too, but it’s choice of borrow checker over GC amounts to trading off a lot of productivity in the general case (but it makes sense for Rust’s goals!).

And yet Go has delivered very useful software used by millions of people.

So has C, and most other popular languages. Doesn't mean they don't have issues, often dangerous issues.

Quick correction for anyone searching: it's `tokio`, not `tokyo`. [0]

[0]: https://tokio.rs/

I find it really curious how people consider "async" as a "basic thing". Coming from C++, asynchronous runtimes were a thing companies and large open source projects were built around. It's not at all a basic thing.

I'd argue it's almost certainly something that should NOT be in the standard library. If anything, the feature was added to Rust too quickly and has way too many rough edges.

When performance and correctness are paramount, I personally stick to Ada/SPARK. Proven to be perfect for the job, and to be honest, Rust is quite a difficult language and I am not sure it is worth learning it considering that Ada/SPARK ticks all boxes when it comes to both low-level and high-level programming for critical systems.

I have fond memories of using Ada in college. Unfortunately it didn’t have much of an open source footprint and the community was downright hostile toward newbies (more so than other communities that I had experienced). I’m sure Ada has the stronger story for real-time embedded systems today, but (as I have learned Rust in the interim) I look forward to the day when Rust breaks into the real-time embedded space. In particular, I’m excited about ferrocene and the broader Rust certification efforts. I would really like to write embedded Rust professionally, and I hope certification will boost the number of embedded Rust jobs (no the to Ada, I just don’t particularly want to re-learn it).

My favorite thing about learning Rust is how helpful the community is.

Do you work in Ada/SPARK professionally? Curious what industry if so.

I think this comes with the caveat that there are a lot more mature libraries for Go just as a function of the age of the language.

Agree, you can see this in https://github.com/rust-unofficial/awesome-rust vs. https://github.com/avelino/awesome-go

I've noticed however that there has been an uptick in great libraries over the last 2 years, with examples like pola.rs, rust-bert, tokenizers etc. starting to build momentum in the ecosystem.

That might have been true 2-3 years ago but nowadays whatever I can think of, Rust already has a library for it.

Most recently I've run into Hyper not having tracing support