That is certainly false hyperbole. There a lot of very useful new features in 2.10, and many people are currently successfully developing in Scala, even with its relatively slow compilation speed.

The Scala team knows compilation speed is important; they have dedicated people working on it full time. This release they made progress by creating an incremental compilation system called Zinc that is shared by Eclipse, maven, and most recently, IntelliJ. They have also said that there is no easy fix on the horizon. Improvements will take time and likely continue to be incremental in the near future.

Actually, the IntelliJ team does not use Zinc. Their compilation subsystem is basically an equivalent re-implementation of Zinc.

The reason they built their own is because they need more data about the compilation process than Zinc provides.

That is partially correct. Intellij doesn't use zinc, but it uses the same compilation system that is at the heart of zinc. From http://blog.jetbrains.com/scala/, there is a section showing how their new system relates to zinc.

It would have been more accurate for me to say that this release the Scala team developed an SBT incremental compiler that is used by Eclipse, Maven, and, most recently, IntelliJ.

Performance problems like these early on in a product's adoption (yes, I know Scala isn't exactly new, but it still isn't widely used) can really give it a bad reputation that is extremely hard to shake.

I hope that they don't fall into the trap that the Mozilla folk have with Firefox. Although Firefox is still slow, it isn't as bad as it once was. Even after substantial effort and some moderate improvements, it still has a very bad reputation. I don't think it'll ever overcome this.

The same could easily happen to Scala, if they aren't careful.

We only run into big compilation speed problems when we have the following:

1) Code that is lumped into one gigantic JAR instead of being broken down into several different libraries that can be compiled faster.

2) Code that forces the compiler to "solve" many problems in the type system, such as recursion and many other powerful type level constraints.

The first isn't a problem with the compiler, it's a problem with how you're laying out your code. Luckily we don't have this problem.

The second is a problem if you've decided to use a lot of type level programming all over the place. I can't say if this is really a "problem" or not because it depends on your use case.

Scala's "customers" (coders) don't really care about why compilation speed is slow. They just want it to be fast. Compiling Scala is definitely slower than compiling Java. If it can be fixed, it should be fixed. If not, that may be one more reason why Scala won't be the next Java.

Sorry but to me this hears similarly too "coders do not care about why runtime speed is slow, they just want it fast". You cannot get gut runtime performance without understanding the performance constraints of data structures and algorithms you use, Similarly, when using Scala, you must be aware of the performance cost - in compilation time - of the various features and act accordingly. So yes, what can be fixed should be fixed - but I am happy with having the possibility to trade ocassionally a powerful feature that will make my coding simpler for some unavoidable compilation delay. (Though I would of course prefer the compilation to be instantanous.)

This argument seems strange. When people complain that tests take a long time to perform, the argument usually falls back to "if your tests are useful, the time spent running them is well spent." Seems the same applies here. Scala takes a long time to compile if you lean heavily on the type system. This makes sense, since the type system can be seen as a type of test system.

So, if you are making useless types, the speed concern is likely best addressed by moving stuff out of the type system. If the extra time is preventing bugs, then few would argue it is wasted.

Or am I completely off in my view?

Typechecking shouldn't take much of the time. I'd wager (having never even looked at the Scala source code) that the time consuming part is compiling everything into JVM bytecode.

Since the type system in Scala is popularly touted as Turing complete, I think you are mistaken in this wager. I remember in one of the exercises for the course, I did something where compilation never terminated. Changed the type of a single function, and compilation took less than a second.

> Since the type system in Scala is popularly touted as Turing complete

So is GHC's. Turing-Completeness says nothing about speed, only about termination prediction. GHCi can type check most codebases in much less than a second. The puzzling thing about the apologism for Scala's slow speed is that some people seem to think that no FP language has ever had a fast compiler before...

Meh, the puzzling thing about many acolytes of anything in programming is the belief that all things have been solved. Usually best in what they care for most.

Clearly, since my point was easily shown as a non-sequitor, I'm not an expert. Following the folks involved with Scala, on the other hand, I don't have any reason to feel that they are the rank amateurs that they would have to be to be to produce such a slow compiler with no need.

My hunch, the support of nominative subtyping is the culprit. My backing for this? Pretty much nothing other than snippets of posts others have made.

I am not sure if that is correct. When I took Odersky's Functional Programming with Scala class last year, it was recommended that we use the Eclipse based Scala IDE that had live worksheets: save the file and everything on the worksheet gets executed (quickly). I am an old Lisp programmer and the Scala IDE worksheets had the goodness of working in a repl. Compiling homework assignments seemed fast also, but that was only a few hundred lines of code.

BTW, off topic but I would really recommend Odersky's class if it is offered again. I mostly use Clojure now, but the material in the class was generally useful for doing functional programming.

Just use sbt ~compile. It will compile every time a file changes (only what is needed) . I've been using Scala full time for a year now, and I've never been bothered by compiling time.

Bullshit ;-) In theory that's how we'd like it to work, but just based on the SBT author's winding explanation of the under-the-hood mechanics (http://www.scala-sbt.org/release/docs/Detailed-Topics/Unders...), needless to say it's a very complex process, one that is far from exact in terms of user expectation (read: surprising). Also, If you use a framework, you're at its mercy. Take Play's routing, for example. Add a single route and EVERY route is recompiled, which means your controllers and, god forbid, their dependencies get thrown into the mix. If you're careful you can minimize the damage, but let's be honest, barring simplistic apps, you're in for ride, a few seconds here and there add up over the lifetime of a project.

It hasn't been a real problem for me. But I cant say the same for every kind of program developed in Scala. I'm not saying that it is not a problem and I dont recognize it. Its just that, for me, the problem is much smaller than some people here seem to think.

Maybe I've waited too long too many times for gcc to compile C/C++ that when I went to Scala I thought waiting some seconds to compile is okay =P

SBT provides incremental compilation, so it's not really all that painful provided your using SBT to build your project. Additionally, your getting a hell of a lot of work out of that compiler.

I fell I should link to Martin Odersky's answer to that remark: http://stackoverflow.com/questions/3606591/why-does-intellij...

Scalac 500-1000 lines per second according to the link, with a startup time of about 4-8 seconds. Startup time can be worked around with zinc/fsc/nailgun

I've used Scala on several (largish) projects using Eclipse and, thanks to incremental compilation, I've barely noticed compile times never mind had it become an issue.

sbt's incremental compiler is a huge improvement over restarting the compile from scratch each time.

Use "sbt ~compile" or "sbt ~test". IntelliJ 12 + Scala plugin even an external running sbt incremental compiler to do compilation for the IDE.

Just make sure you don't ever use incremental compiling to build production jars. There are some edge conditions that exist in both scala and java that cause wrong behaviour to occur. It has to do with information lost when the compile occurs. A full clean and compile produces correct results.

Can you provide more info?

I have 2 scala programs in production, one of which also has one java file (merssenetwister inlined) and I deploy incrementally built jars without any problems so far.

What scala+sbt versions (or whatever) do you use?

(editted formatting)

Here's one example, in order, w/incremental compiling enabled in Java:

Class A contains public static final String x = "x"

Class B contains public static final String y = A.x

Class A modified, A.x = B.y

Result: A.x and B.y = "x"

clean and compile, no code changes

Result: A.x and B.y = null

The reason this occurs is because information is lost during compile. The compiler optimizes the references away and doesn't fix them w/incremental an compile. This occurs in both sbt referencing javac and the eclipse incremental compiler in Eclipse IDE.

While this is contrived, the main point is that more complex interactions occur where these references may not be updated properly and you may end up with behavior you were not expecting due to these optimizations. Interestingly, I think the problem can be solved if the compiler were to keep incremental metadata explicitly stating where these dependencies exist. This certainly is the case with eclipse, where the IDE knows about the link, but still produces behavior that is dependent on compile order and not on the code change itself.

Interesting, I never had this happening to me, and I have not tested your example, but I'm bookmarking this to try some time later. Can you confirm this is a problem with sbt 0.13?

Thanks for taking the time to make and example.

I suppose the term "wrong" I used is not the best, based on what I mentioned. Perhaps "undefined" is better. :)

I always assume undefined is wrong.

Here is more info on using SBT with IDEA 12: http://blog.jetbrains.com/scala/2012/12/28/a-new-way-to-comp....

Here are some pointers about writing code that works well with SBT incremental compiler: http://www.scala-sbt.org/release/docs/Detailed-Topics/Unders...

Following that advice will drastically speed up your compile time while working (obviously, a clean compile will take as long as ever).

I write production code in Scala on a daily basis and compilation time is a non-issue for me - the project is compiled in the background (by Scala Eclipse plugin) so fast that I rarely have to wait for it at all.

And yet people -- thousands of them -- do manage to get useful work done in Scala on a daily basis! It's almost as if Scala's compilation speed is fast enough for most people. :shakes head:

Everybody privately bitches, and some publicly, about scala compile times -- it's slow! Even incremental builds can be painfully slow when you change a file that then triggers the recompilation of a host of other dependent files. Scala is both completely & utterly awesome and not at the same time, impressive, I realize this truth daily ;-)

And yet people -- hundreds of thousands of them -- do manage to get useful work done in Java on a daily basis! I do not want to be one of those people however because I do not like suffering.

I remember various Pascal compilers bragging about 30k lines/s back in the 80's/early 90's.

Are you sure that wasn't lines per minute?

Actually, now that you mention it, yes, it probably was per minute. But still, that was on 12MHz 286's and 8MHz 68k's...

How slow is it (for a moderate sized programm - simple text editor for example)? Minute? Five minutes? Seven? More?

I like this "problem" because it encourages small programs. Also, Scala's type system is just very complex.

If you have a 2000-file program with cyclical dependencies you are screwed. If you use Scala-IDE (which has an embedded sbt doing incremental compiles, which become non-incremental if your dependency structure is fail and you have a huge SCC) on such a project you are doubly screwed. That's not a problem with Scala. Bigass programs aren't a good idea in any language. The programmer-to-program relationship should be one-to-many (for engagement, productivity, modularity, code quality) and not the "enterprise" reverse.

I'm sorry but this is possibly one of the worst comments I've seen... There is a conservation of complexity that you just can't get rid of in an application. To suggest that Scala encourages small programs, you are also suggesting that Scala is only appropriate for small problems, and thus it isn't a scalable language.

See: http://c2.com/cgi/wiki?ConservationOfComplexity

Ultimately, the baseline requirements drive the minimal complexity of code. Sure a poor programmer can drive up the complexity with unnecessary constructs, but the converse is not true: a great programmer can drive to simplicity. For example, a flight control system for a 777 can never be reduced to a single file of 700 lines of code. It will likely always be 2000+ files, and have lots of inter-dependencies.

So I guess (run time safety issues of Scala aside) Scala is not an appropriate language for complex problems by your own words?

Scala's compile speed (with incremental compiles) is generally acceptable if you have reasonable dependency patterns. If you have cyclic dependencies, it's a problem. However, cyclic dependencies are signs of things that are wrong in development practices that can't be blamed on the language.

You mentioned flight control systems: those are unlikely to be written in Scala, being real-time systems, but if one were, it would not need to be compiled and recompiled on the fly.

Scala's a great language but it's not the right language for every problem. No language is.

Good non-answer.

For many large, non-trivial systems have several requirements: - tends to have complex dependencies - needs fast builds

This was a classic: http://www.amazon.com/Large-Scale-Software-Design-John-Lakos...

Essentially how to design C++ systems so that incremental compiles don't sink you. Developer productivity is important.

Scala is badly failing on this front IMO, it reminds me of the old bad days of C++ in a major way.

Using slow compilation speed to encourage small programs is a terrible "solution".

It's slow because it's doing a lot.

If you don't want a cutting-edge, powerful static type system there's an awesome JVM Lisp (Clojure) that I recommend highly.

GHC does a lot of the same things (arguably more stuff) and it's extremely fast, especially if you don't use -O*.

Anyway, type-checking is probably the fastest part of compilation so there's no reason to give up Hindley-Milner for speed reasons. (One of the best features of GHC is that you can use GHCi to typecheck)

GHC doesn't do subtyping. I suspect that is the main reason why Scala is slow - it's certainly the reason why the code is more complex.

OCaml does objects and subtyping, while still managing to remain quite fast.

http://caml.inria.fr/pub/docs/u3-ocaml/ocaml-objects.html

OCaml doesn't have Haskell's type-classes or Scala's implicits. Scala's subtyping is also nominative (versus structural) and it's basically impossible to do type-inference for nominative subtyping. Whether this is good or bad, the fact remains that Scala runs on top of the JVM while Ocaml doesn't. If you ever work with Ocaml you'll notice that the type-inference system breaks down when you're working with OOP and in Ocaml you're really dealing with 2 type-systems shoved in the same language.

What Ocaml does really well, compared to both Haskell and Scala is type-inference, but it's kind of a weak language IMHO as far as functional programming is concerned.

I also don't get all this complaining, because in practice Scala compilation speed is not a problem. After working with Scala in production for a couple of months now, I'm beginning to suspect that all this is just hyperbole perpetuated by people that never worked with Scala a single day in their lives.

When I last evaluated Scala, I found the following:

* Launching the interpreter takes 10 seconds; 50 times slower than Haskell.

* Compiling a small module takes 15 seconds; 10 times slower than Haskell.

* Running 100 command-line unit tests on a small compiled module takes 1.5 minutes; 100 times slower than Haskell.

I also found Scala's type system requires many type annotations even in trivial code. For example, here Scala fails to infer the type of `optBar`, apparently because `None` is also a type:

    scala> class Foo {
         |   def setBar(bar: Int) = optBar = Some(bar)
         |   def getBar = optBar.get
         |   var optBar = None
         | }
    <console>:8: error: type mismatch;
     found   : Some[Int]
     required: object None
             def setBar(bar: Int) = optBar = Some(bar)
                                             ^

Workaround:

    scala> class Foo {
         |   def setBar(bar: Int) = optBar = Some(bar)
         |   def getBar = optBar.get
         |   var optBar: Option[Int] = None
         | }
    defined class Foo

Moreover, Scala's implicits can pose a significant barrier to understanding other people's code.

While OCaml's type system may not integrate objects cleanly, in everyday use type annotations are not necessary, perhaps because objects aren't commonly used.

Similarly, while it's possible to confuse Haskell's type inference by using advanced language extensions, in everyday use one tends not to run into such problems.

> Launching the interpreter takes 10 seconds

With SBT you're only going to do that once per work session. Plus the startup time isn't as bad as you make it sound. I've had a much more painful experience with Ruby (Rails).

> Compiling a small module takes 15 seconds

That hasn't been my experience. I just did a test for a project of mine and it took 5 seconds. Without line-counts such metrics are useless (how small is that module anyway?). Also many Scala projects are composed of multiple subprojects that are efficiently managed by SBT, so a root compile triggers compilation for all these sub-modules, but context-switching between them is what increases compilation cost.

> Running 100 command-line unit tests on a small compiled module

When in the world are you ever going to do that? Like seriously?

> Scala fails to infer the type of `optBar`

In practice it's good to annotate the signature of public functions. Haskell developers do that too, because really, when a developer looks at the definition of "optBar" are you seriously going to suggest that they should be looking at other parts of the code to infer what it returns?

IMHO, this is a feature, as what you want is even worse than what you get with a dynamic language, but YMMV.

> Scala's implicits can pose a significant barrier to understanding other people's code

I haven't met a single library or project of significant size that isn't making use of implicits in one form or another. Whether they are called global state, singletons or parameters injected through DI, we are really talking about implicit parameters.

The difference is that Scala gives you the means to document these implicits in the signature (referential transparency FTW). They are also statically type-safe, can be overridden and have the extra benefit that they make possible some powerful techniques, like type-classes, something that Ocaml isn't capable of.

I see you're attempting to disregard my results by making assumptions about my workflow. I'm not going to respond to these comments.

> I just did a test for a project of mine and it took 5 seconds. Without line-counts such metrics are useless (how small is that module anyway?).

Obviously, comparing absolute times between two unknown systems isn't going to make sense. I was comparing how long it takes to accomplish the same thing on my machine in Haskell and Scala.

> In practice it's good to annotate the signature of public functions. Haskell developers do that too, because really, when a developer looks at the definition of "optBar" are you seriously going to suggest that they should be looking at other parts of the code to infer what it returns?

I agree, annotating type signatures of top-level functions is a good practice. However, in the above example `optBar` is an internal variable in class `Foo`. The point here is I found Scala's type inference to be incapable of dealing even with trivial code.

> The difference is that Scala gives you the means to document these implicits in the signature (referential transparency FTW). They are also statically type-safe, can be overridden and have the extra benefit that they make possible some powerful techniques, like type-classes, something that Ocaml isn't capable of.

Haskell does type classes without the need for implicits. Also, true referential transparency requires control over side effects, which is something neither OCaml or Scala are capable of, but Haskell is.

> Haskell does type classes without the need for implicits

If you really think about it, type-classes are all about passing around a global vtable implicitly. If anything, Scala's implicits are a superset of Haskell's type-classes and allows for techniques that Haskell isn't capable of.

We could debate all day about what's better. If you're going to make the case that Haskell's type-classes are easier to reason about, then I can make the case that a dynamic language beats Haskell on reasoning about it any day of the week. In the end it's all about tradeoffs.

> true referential transparency requires control over side effects, which is something neither OCaml or Scala are capable of, but Haskell is

With all due respect, that's just bullshit. First of all because there is no such thing as "true referential transparency", second because most techniques that are available in Haskell for dealing with side-effects are also available in Scala. As examples, Scala's library is filled with monadic types and the Play framework makes use of Iteratees for composing HTTP responses, something which makes Comet/WebSocket a peach to deal with.

> With all due respect, that's just bullshit. First of all because there is no such thing as "true referential transparency", second because most techniques that are available in Haskell for dealing with side-effects are also available in Scala.

A referentially transparent expression is guaranteed not to have side effects. A Haskell function which doesn't advertise in its type that it may have side effects, by being in the IO monad, can be trusted to be referentially transparent. This cannot be said about OCaml or Scala functions.

Pardon me...are you saying you want something in the type signature of a scala function that guarantees said function will not do input output to files sockets etc...and does not make a call to random number generators and does not modify global vars...so basically all it does is operate solely on params passed to it...absolutely no side effects...is that what you are asking for? That would be awesome from a testability standpoint...

Yes! This is exactly what you get in Haskell, and it is awesome for a number of reasons. Check this out:

http://learnyouahaskell.com/input-and-output#hello-world

Yeah I know exactly what you mean in Haskell. Its just that in scala given the java inter-op there are a whole hunch of escape hatches that will always be there. I remember in the coursera exercises when you submitted code that used mutable vars...it would flag your code and deduct points. So essentially even if someone provided a purefn annotation...to actually enforce that, the compiler would have to scan each function and disbar i/o, calls to rng, calls to global vars... tall order.

Which coursera course? For Scala, or functional programming, or something else? Do you have a link to it?

There's only one so far: FP in Scala, run by Odersky himself.

> If you really think about it, type-classes are all about passing around a global vtable implicitly. If anything, Scala's implicits are a superset of Haskell's type-classes and allows for techniques that Haskell isn't capable of.

First, I'm not a Haskell expert, but from what I understand there are no vtable passed around nor attached to "objects" in Haskell.

Type-classes are used to find an implementation given an instance. Without extensions you cannot do this:

> lst :: Show a => [a]

> lst = [1, "hello"]

So Haskell isn't the slightest dynamic here.

In practice (I write Scala every day) compilation speed is a problem. I just recompiled a work projec, just under 28kloc. 124 seconds is a problem.

Admittedly, we do make heavy use of the type system.

The reason for why it hasn't been a problem for us is because we've been making heavy usage of SBT's support for sub-projects. Scala makes it easier than other languages to decouple components, so after a while you begin to spot possible sub-projects all over the place.

The project I'm working on right now is about 15kloc, but it's split in 3 sub-projects. Coupled with incremental (and continuous) compilation, this compilation speed issue hasn't been a problem for me.

Also, if you look around, many open-source projects built with Scala follow the same technique. It's quite sane too.

Out of curiosity, how much of a problem is it? Is incremental compilation noticeably slow or is it just the full compile?

It's friction in the development process.

Incremental compilation suffers from the fragile base class problem. In a well structured app with proper encapsulation and rigid (and type annotated) interfaces, it helps a lot.

How many times you do full recompile during a day? I'm curious because in my workflow I rarely do full recompile.

If your project isn't properly encapsulated, incremental recompilation will often come close to a full recompile, or at least recompilation of a big chunk of code.

For example, if you delete AbstractFooBase.unusedMethod, sbt will recompile every file everywhere that references a concrete Foo instance.

(In principle it should only need to recompile files which reference unusedMethod, but it's not that smart yet.)

I wonder if those implicits in Scala makes the compilation non-linear in time.

Haskell didn't build on top of a decade-old enterprise language.

Both Haskell and Scala have some excellent work behind them, but they were built under different constraints.

On the other hand, if you do want a cutting-edge language with a powerful static type system, and a compiler an order of magnitude faster than Scala's, you could try Haskell.

Not likely to be fixed any time soon. Does incremental compile not help?