As someone belonging to that group, I agree. short programs are easy to write in Go. If I need to write some utility script I would do it in Go (or more likely python). However, if I need to write a whole new thing I find it very easy to design in Rust in a modular way.
Similarly, if I have a Go and a Rust codebase in front of me, and I need to make changes, it takes longer to work with the Go codebase for nontrivial changes. But this is less stark a difference than the above.
I will say that "modulo compiler perf" is an issue. I was tinkering with the Go compiler and the fact that I could try things out quickly was a big boon. However, I had to spend time figuring out which interface goes where (the documentation doesn't show interfaces implemented on types, you have to figure it out by looking at the source -- this is something that could be fixed though), so it might even out.
But yeah, fixing errors is not hard in Rust (nor do I often get borrow errors, usually it's other stuff), and in fact I find it easier to do than Go because Rust provides extensive errors with suggestions as well as showing the erroring thing inline, whereas Go just points out a file and line number (Again, this could be fixed).
My productivity has been greatly improved by restricting myself to `cargo check` when writing code. It only does type and borrow checking which is much faster than doing all of the LLVM passes (or whatever rustc does after those passes, on that subject I am ignorant).
I've found that when using Atom with linter/linter-rust installed and setup with cargo check I rarely context switch while waiting for errors/warnings to be highlighted (and this is on a 1.6Ghz Xeon machine at work). But maybe I'm just OK staring blankly for slightly longer than others :).