B) Precisely, unlike Haskell, Scala doesn't force you to write "elegant code", you can even use variables. Purists will flame you for that, but sometimes it's the best way to write a small piece of code and as soon as no state leaks outside of the function, it's not that bad. That said, I believe most of the features of Scala make maintenance easier, including the absence of null-checks as it moves errors from runtime to compile time and make major modifications without regressions easier.
The only difference worth mentioning is that they type system won't allow "leaks".
Me too. Luckily this is a false dichotomy.
def :+[B >: A, That](elem: B)(implicit bf: CanBuildFrom[Repr, B, That]): That
This is from the standard collections library.
In Java, I've got a List.add function. In case it's not clear from the name, 'add', I can click through to the implementation and it's pretty obvious what's happening.
In Scala, I've got +, ++, +:, :+, and a bunch of other nonsensical bullshit, and when I click through to the implementation? Even less sense. Whenever I use a standard scala collection, I have no idea what it's actually doing. Additionally, everything favors allocation-happy overly-clever immutable wrappers rather than a simple ArrayList which will smoke those immutable implementations in real-world performance.
The cure is far worse than the disease, here.
Scala's collections have some quirks, but not as many as .NET's collections and you're actually comparing apples to oranges, because you won't find the equivalent of an "add" that returns a new collection instead of modifying the old one in .NET.
This is what happens when you pass judgement unto things you don't understand. Working with immutable data-structures is really, really awesome and Scala's API for these collections is very friendly and very type-safe - as in, if you feel the need to use `isInstanceOf` / `asInstanceOf`, then you're probably doing something wrong ;-)
And I really wish that C# would grow up a little in this regard, as modern programming languages need immutable collections as well, with a nice API to go along with it. And btw - working with Option is super awesome, no category theory needed.
That said, as I was saying in another comment, I'm really excited about this announcement, because this is mostly about the runtime, not the language. You can run things built with Scala on top of .NET right now by means of IKVM. And the JVM finally has some credible competition.
http://msdn.microsoft.com/en-us/library/dn385366(v=vs.110).a...
I think it's kind of the point if the judgement is on the question "what is easier to understand". I think this is the sentiment many people staring to learn Scala are feeling - the barrier to entry, even if you are coming not from the blank slate but from the background of programming many years in many languages, is pretty high. It's not the judgement on "whether Scala and its collections are good/done right", which is entirely separate question from "whether it is easier for someone to understand how C# collections or Scala collections work".
>>> And btw - working with Option is super awesome, no category theory needed.
Well, if you want to do something like making a function that works on Option from a function that works on the underlying type, you pretty soon find yourself in that general area.
As far as not understanding how awesome immutable data structure are.. I'd take a step back before you make assumptions about what other people understand. Do you know anything about cache hierarchy and memory models on modern CPUs? Performance is important to some of us.
Yes I do. Worked 3 years on a soft real-time system with massive load, profiled the shit out of everything. There's an interesting discussion we could have about when immutable data-structures work best, when they've got problems and when it doesn't matter, especially given the extra benefits in dealing with accidental complexity. This isn't the right place though.
> Performance is important to some of us
Yes it is, but performance problems are fixed by means of profiling and optimizing the bottlenecks. Even in a system that has massive load, in many cases in doesn't matter and in some cases immutability increases performance by eliminating contention on reads. And seriously, most people invoking performance problems are not having those performance problems to begin with, therefore my assumption.
As I'm sure you know, when it comes to real-world situations, reducing inter-thread communication and isolating anything mutable is the key concern. That's why I find immutable wrappers that mock mutability to be a bit of a sideshow. You shouldn't have read contention with locking in the first place unless it's for a very good reason.
True, but you know how it is in practice :-)
For example I found that using persistent data-structures work best when you've got single producer, multiple consumers scenarios - so you mutate some state and you want to signal it over asynchronous boundaries to multiple consumers. With an immutable data-structure you just signal it, worry free and then you can keep on changing that state, completely non-blocking / wait-free and with good algorithmic complexity.
Actually non-blocking logic becomes really easy, as you can always shove an immutable value into an atomic reference (note - I'm not saying "wait free", which still takes a lot of work :))
So really, persistent data-structures are great in a multi-threading context, as long as you don't have multiple producers pounding on the same reference holding such an immutable value - if you do that, things can get bad, when compared to specialized concurrent mutable data-structures - because a good concurrent data-structure is able to distribute the contention in multiple buckets instead of just one. But then again, having multiple producers pounding on the same resource is just asking for trouble and has to be avoided, because Amdahl's law.
Also, as you've hinted at, the problem with a normal linked List is the level of indirection. And in general, persistent data-structures imply the usage of trees, which also implies indirections. More advanced persistent data-structures are much better than the linked list is and this is an active area of research, but on the whole there's still much room for improvement.
On the other hand, in my opinion when speaking about performance, the first problem one has is to actually use the available CPUs (e.g. getting CPU usage over, say 70-80%). Which usually is hard to achieve if you have a combination of CPU-bound and I/O-bound tasks and your I/O stuff is not asynchronous. Only after that you can then move on to optimizing the memory access patterns for cache locality and for minimizing the stop-the-world freezes.
Speaking about GC, that's another topic - persistent data-structures have a tendency to generate junk that is neither short term or long term and that invalidates the assumptions that current GCs are making. The JVM at least has really good GCs, but without paying for a pauseless one (like that one from Azul Systems), you can still end up into trouble if you don't pay attention - but then you fire up YourKit's Profiler, find the source for those STWs, optimize and it works out well.
All in all I encourage everybody to find a good library that implements persistent data-structures and integrate them in their toolbox.
The one thing I disagree with, in many server applications, is using the available CPUs is pretty easy. You've got thread pools handling various tasks, just crank them up. In JVM-land, a very heavy 512kb stack per thread is still not really much penalty to pay as long as you re-use them. Aggregate application performance then becomes a matter of completing tasks faster while creating less garbage.
So it all comes down to what you consider a 'task' and how you handle the handoffs between them. The architecture decisions at this level dwarf the improvements from using an array vs list, as you implied, but they also make the usage of immutable types somewhat irrelevant IMO. Seal off mutable code within single-task boundaries and it doesn't matter how ugly it is, as long as you're passing immutable types (just plain javabeans with final members are fine) between boundaries.
Anyways, just my opinion. Great comment.
Why the atomic reference here? I know that provides CAS but if we're talking about a single writer aren't you okay to just replace things anyway?
Thanks!
Gotta test for any chinks in the armour.
Yes.
> Performance is important to some of us.
... and in those cases, you don't have to use data structures that model your problem domain poorly.
The problem isn't "immutable data structures" or "theoretically robust" languages. The problem is finding ways to express computations and their constraints in a way that can be efficiently modeled for your problem domain.
So what will probably happen is that once you see your performance problems, you can pray that it's only some hot spots that you can then replace with faster code - often it's not, so you have 100 places using around 1% of your time budget for example, which is when you can go and start over.
Admittedly not part of the core library, but installing a NuGet package is pretty darn easy.
>>This is what happens when you pass judgement unto things you don't understand ;)
[edit: Guess I should've refreshed the page to see the prior response before writing this. Apologies.]
You're rarely actually going to _see_ that signature (the docs actually simplify it for you), and in practice, it's completely meaningless to 99.9% of Scala you'll ever see.
You might as well consider it pixie-dust.
+ for one, ++ to add a collection, the others are generally going to apply to Cons-like. It takes all of a couple minutes to let these sink in.
Use a mutable.MutableList if that's what you want. Or just use ArrayList.
All this is about as pure FUD as I've ever seen...
Your typical Java developer is used to just look inside the code of whatever library they are using, and find a very straightforward implementation. Scala collections avoid a lot of boilerplate with canBuildFrom, SeqLike and suck, but simple and straightforward they are not. It takes quite a while before it stops reading like Japanese.
And IMO yes, a lot of symbolic methods in collections make relatively little sense. Don't forget that list also has ::, :::, +:, :+ and :\. There's more than a few, and there are no textual versions of them, for those that don't have them all memorized. They are a bit of a relic from the time Martin thought that /: was a good idea. It's fortunate that now only the scalaz people keep doing such things, because excessive use of symbolic operations hurt language adoption.
And he doesn't even get into other early confusion points, like how we have =>, <-, and ->, or how decomposing Seqs is not exactly pretty. Last week I had to help a guy that had been using Scala for 6 months to understand the 'punched in the face' operator :_*
So no, it's definitely not FUD. Are they issues that hurt my day to day Scala use? Not at all. Scala is my favorite language. Being able to use it instead of Java or Clojure is worth a good 15K a year for me.But that doesn't mean that I have forgotten some of the little things that made the learning curve tough at first. Thanks the heavens that I managed to end up finding a Scala job where I could learn from one Bill Venners.
Everybody decried Ruby's 107 methods on Array. Then Fowler came out with "fluent-interfaces", and how often do you see someone make the claims that Ruby's Array is indicative of a general badness because it has a lot of methods and you can't memorize them all in an hour as a newbie?
scala.collection is the same deal.
:\? Sure it's not a good idea. I wouldn't debate that. Who uses that? foldLeft/foldRight.
And what's the deal with trying to memorize the entire interface anyways? 8 or so years with Ruby, writing libraries with over 4 million downloads (https://rubygems.org/profiles/ssmoot), and there are definitely methods in Ruby's Array I'm unfamiliar with.
So what?
While :: and ::: look a little foreign, I don't think asking people to learn them if they want to work with Lists in a Functional manner is anymore difficult than learning what the "spaceship operator" does in Ruby. And it's optional. You don't have to use them. But they're usage will probably be the smaller part in the grand scheme of things. Pattern Matching and accomplishing functional recursion with immutable data is the bigger picture. Outside of that context (and the REPL I guess, for convenience) you're just not going to see either of those operators very often (IME).
You don't use a "splat" (aka 'punched in the face'? That's a new one to me) operator very often. It's actually one of the few semi-pattern-matchy areas of Ruby so it comes pretty naturally for me.
You don't become a pro overnight. You can trust I'd be teaching infrequently used idioms in Ruby to a developer who'd only been using it for six months. Been there, done that. ;-)
I guess where I'm coming from is using CBF to claim Scala is a confusing, indecipherable language. Everyone's had that argument. I don't know that anyone wants to stand up and claim it's the best. But it works. Actually using it is a non-issue since you don't actually explicitly use it. And look at the docs. They're actually pretty great overall.
It just bothers me I suppose that someone interested in exploring Scala would be dissuaded by something that's only ever been a problem for 1% of 1% of Scala developers.
If you're just looking to swap in Play to replace Rails, odds are you'll never run into any of these CBF "concerns". At all. That's the very definition of "FUD" IMO.
For every CBF in Scala there's a Calendar in Java. Languages aren't perfect. CBF is probably a wart. But it's a hidden one. If you let it scare you off from Scala that's sad, because it really has about as much to do with day to day Scala development as array.c (https://github.com/ruby/ruby/blob/trunk/array.c) has to do with Ruby development.
That line alone drives home the point for me, you can read the rest of scala.collection.immutable if you need more convincing.
Also, idiomatic Scala code is perfectly readable. You have no right to complain about readability until you've dealt with spaghetti code written for Megacorp Inc, that completely ignores the fundamentals of structured programming. :)
I'm interested in the Haskell part. What features do you think are fundamentally misguided ?
But, for example, https://www.haskell.org/tutorial/io.html.
At the end of the day, this is a huge inner-framework anti-pattern over the same procedural syscalls that every other language handles procedurally. I shouldn't have to care what a monad is, and side effects? The whole point of I/O is side effects. It could be a no-op, idle process or CPU-burning busy loop if I didn't care about side effects.
From that doc:
" So, in the end, has Haskell simply re-invented the imperative wheel?
In some sense, yes. The I/O monad constitutes a small imperative sub-language inside Haskell, and thus the I/O component of a program may appear similar to ordinary imperative code. But there is one important difference: There is no special semantics that the user needs to deal with. In particular, equational reasoning in Haskell is not compromised. The imperative feel of the monadic code in a program does not detract from the functional aspect of Haskell. An experienced functional programmer should be able to minimize the imperative component of the program, only using the I/O monad for a minimal amount of top-level sequencing. The monad cleanly separates the functional and imperative program components. In contrast, imperative languages with functional subsets do not generally have any well-defined barrier between the purely functional and imperative worlds."
So, basically, they acknowledge that their theoretical model has a huge impedance mismatch with what we write programs to do (I/O, eventually, somewhere). And that's fine, they can knock themselves out and I hope it's fulfilling for them. It's not for me.
No, you have a tiny, very simple type that allows for type safe IO. It also happens to make haskell a more powerful imperative language than most imperative languages, as IO actions are first class and can be passed around and manipulated like anything else.
>So, basically, they acknowledge that their theoretical model has a huge impedance mismatch with what we write programs to do (I/O, eventually, somewhere)
No, they acknowledge that doing IO is so important that it should be done correctly. You are going to some pretty extreme mental gymnastics to misrepresent a language you want to hate.
The I/O monad modifies the Universe that contains the set of all functions that comprise your program (if I even understand the concept correctly).
The right half is usually empty, except when you use the `>>=` operator (or flatMap) on an existing action, e.g.
let c = a >>= f
That operator chains the function `f` can take the value produced by the first IO action and produce another IO action. After we apply that operator, the result `c` is a cons cell where the left part is the original action `a` and the right part is the function f which takes the value produced by a and produces the next action. Its sort of like a cons cell in regular lists, except the next value is provided by a function. A lazy cons cell, perhaps :)
The final result is a lazy chain of I/O cons cells (called "main", of course :P). Its passed to the Haskell runtime, which executes that chain as a recipe, alternating between doing I/O actions and evaluating the function to decide what to do next.
So what does this buy us? Mostly just referential transparency. What does referential transparency buy us? Easy refactoring. We can replace any expression with its value, even stuff like `putStrLn "test"`. We can say `let writeTest = putStrLn "test"` at the top of the file then write `do writeTest; writeTest` in main.
Another neat thing is that do syntax isn't limited to just IO, but works with anything that implements `flatMap` (and `unit`, which I forgot to mention). That means we can build our own imperative DSLs that produce IO-like monads which are then interpreted by our own interpreter, and the users of those DSLs can use the same do syntax. Which is pretty awesome. Here is a simple example: https://gist.github.com/tonymorris/b5dba9d7d877051d0164 and a much more complex one http://augustss.blogspot.com/2009/02/more-basic-not-that-any... :)
You can handle every single error explicitly (as in Go) using pattern matching
eitherResultOrError = operation1 arg
case eitherResultOrError of
Left error -> handle error
Right result -> handle' result
let eitherResultOrError = do
x <- operation1 arg
y <- operation2 x + 1
z <- operation3 x y
case eitherResultOrError of
Left error -> handle error
Right result -> handle' result
For IO operations and other monadic actions, its best to use EitherT, ErrorT or MaybeT, which are monads that can add error handling to any other monad. To understand how these work, its probably best to implement MaybeT. Basically, they add another wrapper to other monads to redefine what the bind operator (`>>=`) does
That sounds rather complicated. My way of thinking of it is simpler than that. a -> IO b is just a function from a to b that can do some I/O. Nothing more complicated than that.
Separation of pure and unpure components greatly simplifies the reasoning about the problem domain. Traditional imperative programs are often full of subtle bugs because calling a procedure can have arbitrary effect on your system. For example calling function with the same arguments can return different values based on arbitrary hard-to-track reasons (such as your OS's scheduler). It's just too much details to keep in your head.
But in Haskell pure functions are guaranteed to have same result with same input. It enables the programmer to create logically isolated blocks without messy interdependencies.
It's especially helpful for concurrent programming by freeing you from all the non-deterministic spaghetti.
Haskell has a steep learning curve, but it'll make you a better programmer in the long run.
Sure it does, and composes well under it.
It's late on EST but I promise if you put effort into explaining a higher-order take on this I'll put effort into reading and understanding it tomorrow. Have a good night.
There's no theoretical stuff here, it's just making sure that the caller knows about callees having side effects.
One doesn't have to learn category theory to realize that Options are great, especially when accompanied by a little bit of help from pattern matching, map, flatten and getOrElse.
The fact that many Scala lead figures that come from a Haskell background are bringing with them a holier than thou attitude doesn't mean that there aren't some functional concepts that are very useful even if your code is mostly imperative, and Option is arguably the least controversial of the lot. It's so uncontroversial it's in Java 8, although suffering from the fact that it is lacking some of the great Scala goodies.
To make Haskell a serious advantage and not just a minor benefit (compared to, say, SML), you probably also need to understand kinds. Let's bear in mind that the vast majority of practicing programmers can't reasonably define an algebraic data type, and that it's not their fault, because the ROI for learning such things is often negative.
I do think there is a reactionary "don't-wanna-learn anything" vibe against Haskell among certain groups. But I think we should be clear that, to get the power that Haskell promises, you do need to learn many new things. Those things, while not exactly category theory in a narrow sense, are closer to category theory than they are to conventional programming knowledge.
data Bool = True | False
There are a few reasons for this that get dismissed by the day to day practitioners. First, there is the cognitive rewiring required to think of everything as an inert expression. In Haskell there are no actions, just descriptions of actions that the runtime manages. I'm specifically talking about I/O and its monadic implementation. Second, learning about monads is not enough. When effects are encapsulated as monads you need to understand monad transformers to fruitfully combine effectful computations. This is by no means the best way to do things because there are also implementations of effectful computations with row types and extensible effects, e.g. PureScript, that requires a lot less cognitive overhead and is less error prone. Third, many of the high-powered libraries in the Haskell ecosystem are so heavily reliant on categorical constructs, e.g. free (co)monads, functors, applicatives, monoids, bifunctors, Kleisli categories, etc. that getting through all that thicket to be truly productive with the libraries instead of just copying and pasting requires a time investment that is of dubious value to many programmers and you're better off learning about security practices on OWASP because you are more likely to encounter a SQL injection than you are to encounter a Kleisli category of a monad.
I'm not saying knowing these things is not useful or won't make you a better programmer but to just demonstrate that there is indeed a cognitive overhead that might not be worth it. I like category theory as much as the next mathematician but programming with categorical constructs is not necessarily the most optimal way to do things when all I need is a screen scraper for an XML feed.
Perhaps they get dismissed because we all went through the process of learning haskell in order to become day to day practitioners, so we know these reasons are made up nonsense.
A), it was an uncalled-for digression into language wars, and
B), now that we're here anyways, a lot of us think the majority of features in scala and Haskell for that matter are fundamentally misguided. My goal is not to write elegant code, but to write the least complex code with the lowest cognitive overhead. TCO, 10x the man-hours in maintenance and all that. If the choice is between having to write null-checks or having to understand category theory to read my code, I'll take the null-checks.