News for hackers. Stuff that matters.
Re:The struggle with Rust by rbehrends on Tuesday January 24, 2017 @09:06AM (#13471346)
This is a different issue. The problem that the author is observing is that Rust operates on the hypothesis that the ownership relation is mostly left-unique and acyclic and that you run into problems when this hypothesis doesn't hold. And this affects not only explicit data structures, but also the implicit structures that involve stack frames (local variables, closures, etc.).

While Rust has mechanisms to deal with this (copying, Rc<T>, etc.), all these mechanisms add pain points, and for problems that are not just theoretical. It affects a number of design patterns, closures that survive the scope they were created in, functional-style programming, functional data structures, any data structure that naturally involves a DAG or cycles.

The difference compared to a (good) static type system is that a good [1] static type system does not measurably reduce expressiveness; Rust's ownership model does.

This is not to say that there isn't a point to Rust's model. Rust directly competes with garbage collection as the main alternative model to achieve memory safety and is an obvious choice in cases where garbage collection is not a realistic option. But where garbage collection is an option, the tradeoffs that Rust makes become much less attractive.

[1] Obviously, primitive static type systems (example: Java up to version 1.4) do pose a problem.

Re:The struggle with Rust by humanrebar on Tuesday January 24, 2017 @09:20AM (#13471463)
First, the expressiveness is all there. You just need to explicitly say you are responsible by using unsafe blocks. So risky pointer and allocation logic becomes the programmer's job, not the borrow checker.

Second, this article only considers the costs of a stricter borrow checker, not the benefits. It will be more work for the coder to have to think about how allocation and ownership is communicates. But I've been through much bigger headaches in production code at inconvenient times (for the business plan) due to dangling pointers, multi-writer concurrency bugs, and ill-defined memory models in software designs.

Even if Rust is much harder than alternatives instead of just different, and even if that's inherent in borrow checking, it's not clear that it's still not worth it in the long run.

Re:The struggle with Rust by Bartweiss on Tuesday January 24, 2017 @10:01AM (#13471793)
> Even if Rust is much harder than alternatives instead of just different, and even if that's inherent in borrow checking, it's not clear that it's still not worth it in the long run.

One obvious argument is that Rust frontloads its difficulty. It's a terrible tool for most 30 line tasks, simply because the majority of your work will go into arguing with the compiler. But that's true of a great many tools - if there's an adequate payoff in safety and comprehensibility at scale, they can still be worthwhile.

That doesn't solve the learning-and-usage problem, though. Most of us are aware that the explosion of dangerous-at-scale tools like Javascript is largely because no one adopts a language that's unfriendly until you reach enterprise sizes. Even if Rust is "worth it", it's unlikely to catch on while the learning experience feels like pulling teeth.

Re:The struggle with Rust by bluejekyll on Tuesday January 24, 2017 @10:42AM (#13472200)
> It's a terrible tool for most 30 line tasks, simply because the majority of your work will go into arguing with the compiler.

This arguing with the compiler is really only while you are learning Rust and it's nuances. For me the first two weeks were this way, then I started thinking more in Rust terms. Now I rarely deal with the borrow checker at all, but I do end up in some complex type situations.

You're spot on with the frontloads difficulty. Honestly though, I can't think of any language that I've learned that in under a few weeks I knew how to properly do things in the language.

Python was this way for me. My code looked like C when I started with it, and then I learned after a few weeks about 'with' and 'yield' and started writing real code.

Rust is worth the initial investment, but the first two weeks will require some perseverance.

Re:The struggle with Rust by spullara on Tuesday January 24, 2017 @11:17AM (#13472574)
I think the difference is that while you are coding Python in a C style the code is working. The front loaded nature of Rust means that you are in a constant fight to get something to run.
Re:The struggle with Rust by bluejekyll on Tuesday January 24, 2017 @11:30AM (#13472757)
Yes, and I understand how frustrating it can be, I had to learn it, too. My only point is that after you spend the time with it, the cognitive load becomes a lot less and you've learned some new things about programming at the same time.
Re:The struggle with Rust by spullara on Tuesday January 24, 2017 @12:47PM (#13473613)
I believe this. However, my concern is that the number of people willing to make it through that learning curve is small and will limit the adoption of Rust.
Re:The struggle with Rust by bluejekyll on Tuesday January 24, 2017 @02:25PM (#13474609)
I'm hoping that things like the Rust Language Server and it's integration into some of the IDE's out there will help with the initial learning curve; already the rustc error output is a million times better than when I started with it.

I definitely share your concern; for me, Rust was like putting on a glove perfectly shaped for my hand. It works and prevents every basic pattern that it took 10 years to learn and start applying to all my code thereafter. If I had Rust all those years ago, it would have saved me a ton of time... but I probably would have turned my nose up at it, b/c I wouldn't have had the experience to understand what it was saving me from.

So yes, I want the learning curve to be lower, but not at the expense of losing all of the safety it brings to developing fast and reliable software.

Re:The struggle with Rust by spullara on Tuesday January 24, 2017 @08:48PM (#13477711)
Have they thought about allowing code to work by warning and elevating the scope so you can quickly get the procedural work done and then attack the memory usage?
Re:The struggle with Rust by bluejekyll on Wednesday January 25, 2017 @12:13AM (#13478748)
I've been thinking about a good answer to this, but I don't know what it would look like. Code looks so different when you need to be concerned about mutable vs. immutable borrows. For example you might want to iterate over a list perform an action and then delete items after doing work. This needs to be broken into two blocks, one that does the iteration and then a second block to remove any elements that no longer belong.

That's an example of just two blocks, but if it's deeper structures then ignoring this until the end could cause you to go back and restructure significant sections of code to make it abide by the mutability rules.

The process I use to stay productive, is to write small blocks of code, write a unit test for it, compile, and then move on. That way I don't build up a ton of compiler debt...

Re:The struggle with Rust by mmstick on Wednesday January 25, 2017 @05:38PM (#13486944)
I've been writing Rust solutions for two years now, and I do not feel like I am in a 'constant fight' to get something to run. That only happened for the first two or so weeks as I was learning Rust.
Re:The struggle with Rust by braveo on Tuesday January 24, 2017 @11:21AM (#13472640)
> This arguing with the compiler is really only while you are learning Rust and it's nuances. For me the first two weeks were this way, then I started thinking more in Rust terms. Now I rarely deal with the borrow checker at all, but I do end up in some complex type situations.

It just means you've learned to head off problems before they occur, that does not mean the problems you're heading off should be there.

Re:The struggle with Rust by nickpsecurity on Tuesday January 24, 2017 @11:43AM (#13472931)
"One obvious argument is that Rust frontloads its difficulty. It's a terrible tool for most 30 line tasks, simply because the majority of your work will go into arguing with the compiler. "

It does. Ada did too with people saying same stuff. Once they got used to it, they swore by Ada since their stuff usually worked after they got it past compiler. I also add that the alternative to Rust for GC-free safety of heap code is essentially separation (VCC) or matching logic (KCC). That's probably much harder for average programmer to work with than structuring stuff to get through borrow checker easily. Cyclone and then Rust were huge steps up on dynamic, memory safety given they're accessible to people who don't know formal verification.

That's the comparison people keep leaving off griping about borrow checker. I think Rust looks a lot better when one adds it. Still a headache but good in relative way.

Re:The struggle with Rust by allengeorge on Tuesday January 24, 2017 @11:13AM (#13472529)
My observation is that that's not quite true. It's fine for smaller tasks, but that mushy middle where you're discovering the parameters and outlines of a problem can be painful. Looser (note that I explicitly don't say "better" or "worse") languages give you more leeway to play around and discover how the problem is structured.
Re:The struggle with Rust by mmstick on Wednesday January 25, 2017 @05:36PM (#13486934)
I can easily slap out a 30-line program and have it compile and work on the first try with Rust, guaranteed. No competent Rust developer has issues writing solutions in Rust. Nobody but newcomers are 'arguing with the compiler'.
Re:The struggle with Rust by rbehrends on Tuesday January 24, 2017 @10:37AM (#13472162)
> First, the expressiveness is all there. You just need to explicitly say you are responsible by using unsafe blocks. So risky pointer and allocation logic becomes the programmer's job, not the borrow checker.

No, unsafe would be overkill and if you needed unsafe, it would put Rust at a massive disadvantage vis-à-vis garbage-collected languages.

Simply put, Rust doesn't handle multiple ownership well. It results in syntactic bloat and/or runtime overhead. But multiple ownership occurs naturally in a number of real-world programs.

> Second, this article only considers the costs of a stricter borrow checker, not the benefits.

This is why I very specifically mentioned that it's a tradeoff.

Re:The struggle with Rust by bluejekyll on Tuesday January 24, 2017 @11:24AM (#13472686)
I think you're implication here is that other languages do handle multiple ownership well. What I think you find in Rust, is that the guarantees you need to make when using unsafe in these limited situations, are the exact same guarantees you need to make in a language like C or C++.

Destroying a circular list takes some thought to make sure you don't double free, same as Rust. Rust just makes those sections of code very explicit and calls your attention to it, while in C you may end up with a segfault, or worse a program that only shows odd bugs during runtime. At least Rust is telling you that you need to put some extra thought in this section.

Re:The struggle with Rust by rbehrends on Tuesday January 24, 2017 @12:21PM (#13473355)
Your assumption seems to be that I'm talking about languages with manual memory management. I'm not; I'm talking about garbage-collected languages, where you get memory safety automatically, regardless of ownership rules. In fact, C++ has some of the same problems that Rust has when it comes to multiple ownership.
Re:The struggle with Rust by kaoD on Tuesday January 24, 2017 @12:38PM (#13473525)
Then use https://github.com/Manishearth/rust-gc if you're willing to pay the price of a GC (still experimental). Or better, stick to a GC'ed language.

If you don't need manual memory management, it's obviously going to be a burden. There are still use cases for manual memory management though, and that's where Rust comes in.

Use the right tool for the job.

Re:The struggle with Rust by Manishearth on Tuesday January 24, 2017 @12:56PM (#13473722)
(Please don't use rust-gc, it's more of a proof of concept. I mean, it works, and some folks do use it, but Rc + Weak are probably better options)
Re:The struggle with Rust by rbehrends on Tuesday January 24, 2017 @12:49PM (#13473634)
> If you don't need manual memory management, it's obviously going to be a burden. There are still use cases for manual memory management though, and that's where Rust comes in.

That's exactly the point that I made in my original comment?

Re:The struggle with Rust by kaoD on Tuesday January 24, 2017 @12:57PM (#13473734)
Sorry, HN isn't great at keeping track who's who on a comment tree.
Re:The struggle with Rust by galangalalgol on Wednesday January 25, 2017 @07:10PM (#13487482)
Does GC imply memory safety? Use after free and dangling references probably can't happen, but I have never heard of a GC supplying cross thread memory safety. The borrow checker does.
Re:The struggle with Rust by rbehrends on Thursday January 26, 2017 @01:14PM (#13493727)
Memory safety in multi-threaded programs is not a hard problem once you have GC. Eiffel solved it decades ago. The JVM also has a weak form of memory safety; it does not preclude data races, but the VM cannot crash as the result of a data race.
Re:The struggle with Rust by humanrebar on Tuesday January 24, 2017 @10:54AM (#13472326)
> No, unsafe would be overkill

I don't see an issue with 'unsafe' if it's an implementation detail and well-tested. Obviously it would be better to avoid it if possible. 'unsafe' is overkill sometimes and worth it other times.

Re:The struggle with Rust by bluejekyll on Tuesday January 24, 2017 @11:18AM (#13472601)
Instead of unsafe, as much as possible should be looking at interior mutability, Cell and RefCell allow for this, with no unsafe, and they guarantee for you at runtime that what you believe is accurate.

unsafe should be IMO, reserved for only those cases where strictly needed, e.g. FFI is the clearest example.

Re:The struggle with Rust by braveo on Tuesday January 24, 2017 @11:19AM (#13472606)
> if it's an implementation detail and well-tested

The same can be said of C and C++, at which point, why does it matter if you use Rust?

Re:The struggle with Rust by humanrebar on Tuesday January 24, 2017 @02:40PM (#13474745)
Rust has opt-out safety with 'unsafe' blocks. C++ doesn't even have opt-in safety. You can get fairly close with smart pointers and some other techniques, but 'fairly close' doesn't cut it when you're talking about memory corruption issues.
Re:The struggle with Rust by AstralStorm on Tuesday January 24, 2017 @03:24PM (#13475111)
You can enforce safety by using the strict type system combined with polymorphic memory allocators in C++17, and get quite close to that with C++11.

The only thing you cannot exactly prevent is "cast_dammit_cast".

Re:The struggle with Rust by dbaupp on Tuesday January 24, 2017 @03:39PM (#13475249)
Could you be specific about how the type system and polymorphic memory allocators defend against things like dangling references, iterator invalidation and use-after-free/move? I can't imagine an allocator affecting these unless it's one that just has a no-op free (which only use-after-free, anyway), nor is the type system strong enough by itself.
Re:The struggle with Rust by AstralStorm on Wednesday January 25, 2017 @02:45AM (#13479543)
1) If you cannot get a plain reference you cannot get dangling references. 2) Use data structures that keep iterators valid and modern reference counted internals. (so that when you have an iterator, you have the object) Alternatively copy on write semantics. 3) Use after move is relatively tricky if you want more. However, your allocator or data structure can catch it with specific implementation of move operators. Use after free can be reduced to use after move or caught with right delete/new operators. 4) Instead, you may have an API where you have your own high level reference-like type. There is a cost to it, same as in Rust. 5) What you cannot do is enforce people to use it always.
Re:The struggle with Rust by dbaupp on Wednesday January 25, 2017 @02:22PM (#13484976)
> There is a cost to it, same as in Rust

All of these things have a larger cost than Rust: you're forced to use pervasive reference counting and very defensive data structures. Rust allows you to get away with pointers deep into other things, as well as aggressive data structures, all in safe code.

Rust also has reliable use-after-move defense for arbitrary resources (not just vague dynamic protections for only the memory resource), and defends against data races. That is, Rust has true safety in a way that can give higher performance than approximate safety in C++.

Re:The struggle with Rust by pjmlp on Tuesday January 24, 2017 @11:31PM (#13478538)
Kind of, you cannot enforce your team mates, consultants and third party libraries that are part of the whole product to use them.

Static analysers and code reviews are not always possible, depending on the target OS and compiler being used, or how the whole project across teams is organized.

Re:The struggle with Rust by jackmott on Tuesday January 24, 2017 @09:25AM (#13471502)
It might be worth it over C++, but assuming the task at hand is not hindered by garbage collection, is it ever worth it over an ML style language?

It certainly takes a lot more typing to get things done.

Re:The struggle with Rust by pmoriarty on Tuesday January 24, 2017 @12:52PM (#13473674)
I think it's always worth it to learn new paradigms. They expand your view of what's possible, and give you a richer understanding of the other languages you already know, understanding some of the tradeoffs they made, and give you other conceptual and practical tools in your toolbox.

It's much like visiting other, radically different countries than the ones you've been brought up in. Even if you don't wind up moving there permanently, you'll be richer for the experience.

Re:The struggle with Rust by rntz on Tuesday January 24, 2017 @10:45AM (#13472229)
> The difference compared to a (good) static type system is that a good [1] static type system does not measurably reduce expressiveness

This is pseudo-jargon. What counts as a "measurable" reduction? Do you have any substantive evidence that "good" typesystems don't reduce it? And why are only type systems which satisfy this property "good"? You talk cogently about tradeoffs later on, but this sentence taken literally suggests only a type system which doesn't make this particular tradeoff can be "good". Why?

My go-to languages for the past few years have been Haskell and Racket. There have definitely been times I picked Racket because I knew Haskell's type system would interfere with my expressiveness. Certainly Haskell is less restrictive than Rust with regards to ownership, but all type systems restrict you. I think GP is spot-on in pointing out that you just get used to it over time as you learn to think and code in ways that avoid pushing against the type-system's weak spots.

> But where garbage collection is an option, the tradeoffs that Rust makes become much less attractive.

Yes, precisely. It's not a fundamentally different issue from getting used to other static type-systems; it's just a different tradeoff.

Re:The struggle with Rust by rbehrends on Tuesday January 24, 2017 @11:10AM (#13472489)
> This is pseudo-jargon. How do you measure expressiveness?

Actually, not jargon. I had Matthias Felleisen's characterization of expressiveness [1] in mind (incidentally, he's also one of Racket's authors).

> Do you have any substantive evidence that "good" typesystems don't reduce it?

If we want to be precise, there is a continuum from statically to dynamically typed language. A dynamically typed language is, after all, the same as a static language with just a single polymorphic type. This means that we can trivially translate a dynamically typed program into a statically typed program, though obviously we don't gain anything by that at this point (this is essentially an argument that Bob Harper has advanced before) and we pay for it in additional verbosity. However, in practice we know that we don't need all that flexibility and can constrain types more, allowing us to get better type guarantees; static types then also allow us to do things that we can't do in a dynamic language (such as compile-time overloading).

[1] http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.51.4...

Re:The struggle with Rust by rntz on Tuesday January 24, 2017 @12:54PM (#13473697)
EDIT: I've been chewing over the following:

> This means that we can trivially translate a dynamically typed program into a statically typed program, though obviously we don't gain anything by that at this point (this is essentially an argument that Bob Harper has advanced before) and we pay for it in additional verbosity.

This is exactly the opposite of the argument I advance below; you're claiming that there is a "local" translation from dynamic to static. And on reflection, you might be right, but I'm not entirely sure. It hinges on exactly what counts as "local". Translating away exceptions, for example, requires inserting case-analyses which propagate an exception if one has been thrown literally everywhere in your program. It's "global" in the sense that it touches every node in the program; it's "local" in the sense that the transformations performed don't involve any global knowledge or analysis. I'm rereading Felleisen's paper now to see how precisely he defines things.

ORIGINAL:

> I had Matthias Felleisen's characterization of expressiveness [1] in mind (incidentally, he's also one of Racket's authors).

Great! I believe, by that definition, your assertion that "good" typesystems don't reduce expressiveness is false, at least for common type systems such as Haskell's, Java's, ML's, etc. There are plenty of programs which require non-local rewriting ("a global reorganization of the entire program") in order to appease the typechecker.

A simple example can be adapted from the expression problem: suppose I'm writing an compiler for a simple language, so I have an AST datatype and some functions on it. Then I add a pass to my compiler that removes a certain feature from the language I'm implementing - a certain branch of the AST. Now I wish to simplify my backend by removing the case which handles this branch of the AST. I cannot do that without either dynamically failing in that case (which a type system might even prevent me from doing) or introducing an entirely new AST type which does not have that branch - but this could require duplicating or else considerably generalizing existing code that operates on the AST (a global reorganization). Of course, a sufficiently powerful typesystem might let me do that, but for any typesystem you name, I suspect a similar example can be procured.

To get more precise, the "feature" we're considering adding is "ignoring the typechecker and running anyway". For example, GHC's -fdefer-type-errors. The question is "can any legitimate program written with -fdefer-type-errors be locally translated into an equivalent program that typechecks without -fdefer-type-errors". Without reaching for an explicit escape hatch like unsafeCoerce (which is analogous to "unsafe" blocks in Rust), I think the answer is no.

Re:The struggle with Rust by dllthomas on Tuesday January 24, 2017 @07:43PM (#13477332)
First, let me say that I think you're right that expressiveness is reduced. In the best case, all of that reduction is of things we don't want to express because they are incorrect - but in practice (and probably in theory!) the boundary will always be fuzzy.

That said, with regard to your "kill a branch in your AST" example, you can kill a branch with Void:

    import Data.Void

    frobnicate :: Either a Void -> a
    frobnicate (Right x) = absurd x
    frobnicate (Left x) = x
Now you can rely on no-one accidentally handing you a Right expecting you to do something reasonable with it.
Re:The struggle with Rust by rbehrends on Tuesday January 24, 2017 @01:58PM (#13474355)
> Great! I believe, by that definition, your assertion that "good" typesystems don't reduce expressiveness is false, at least for common type systems such as Haskell's, Java's, ML's, etc.

Haskell is not very good at extensible runtime polymorphism (one of its weaknesses). Try OCaml's polymorphic variants.

Re:The struggle with Rust by rntz on Tuesday January 24, 2017 @04:54PM (#13475973)
Okay, then I can pick another example: A dictionary where the type of the values depends on the key they're inserted under. (For example, a representation of the closure of a function in a typed language.) Of course, then I can go to a dependently typed language, but are you really going to tell me that only dependently typed languages count as "good" static type systems?
Re:The struggle with Rust by rbehrends on Tuesday January 24, 2017 @08:24PM (#13477584)
I am not arguing completeness here; a type system is not a full-blown specification language and does not have to be. (I actually tend towards the point of view that trying to make them complete specification languages has limited practical value, but that's a different issue.)

The point here is that the type system does not restrict me from implementing a data structure that adheres to this constraint, even if it does not enforce it for me.

Re:The struggle with Rust by dllthomas on Wednesday January 25, 2017 @12:09AM (#13478724)
You can do this in GHC Haskell with existential quantification, or trade static checks for runtime with Data.Dynamic.

(Again, not arguing against your thesis here.)

Re:The struggle with Rust by codygman on Tuesday January 24, 2017 @02:26PM (#13474616)
After quickly scanning https://realworldocaml.org/v1/en/html/variants.html, isn't the same thing accomplished by GADTs?
Re:The struggle with Rust by rbehrends on Tuesday January 24, 2017 @02:39PM (#13474727)
The point of polymorphic variants is that the entire type doesn't have to be fully defined in one place; you can have partial definitions in a module and those definitions can even overlap. You can have (say) [ `Int of int | `String of string ] in one module and [ `String of string | `Char of char ] in another, and the typechecker will handle intersections and unions of these types.
Re:The struggle with Rust by braveo on Tuesday January 24, 2017 @11:17AM (#13472585)
I agree, ayende is a C# person, he's well aware of statically typed languages.

I also agree about your other points, I think maybe one of these days the borrower will become more sophisticated, but these issues he's complaining about are the exact reason I put Rust down. It just felt too sophomoric, as if the language wasn't really ready.

Not all is bad, I really like the way errors are handled in Rust, but there are definitely some problems with the approach they take.