News for hackers. Stuff that matters.
Re:Swift architecture at Uber by hota_mazi on Wednesday April 26, 2017 @11:30PM (#14208983)
> The reason for this is that, as much as I know, that a compiler does type checking for every single file. So if you spawn 200 processes of Swift compilers, it needs to 200x check all the other files and make sure that you're using the correct types.

I'm a bit baffled by that. Is the Swift compiler that naive?

Surely you know how to assess the number of processors/cores on your system and you spawn threads in a way that doesn't lead to diminishing returns. You use a bounded thread pool and you stay within these bounds.

Seeing such a speed up from simply merging text files is really puzzling to me. You have to type check the code anyway, surely the overhead of opening a new file is completely negligible compared to running this type checker on the same file? Especially since all these instances of the type checkers have to share a lot of data anyway.

Swift is very cool and I'm excited to see it and Kotlin become our next generation languages for mobile, front end and back end alike, but it seems to me the Swift compiler is still very immature.

Re:Swift architecture at Uber by mahyarm on Thursday April 27, 2017 @12:29AM (#14209171)
There are a lot of 'obvious optimizations' like this lurking in swift. I get the impression they are somewhat engineer constrained in an interview with chris latner.
Re:Swift architecture at Uber by andrecl on Thursday April 27, 2017 @11:30AM (#14212223)
Is there a link where Latner says the Swift team is engineer constrained?
Re:Swift architecture at Uber by mahyarm on Thursday April 27, 2017 @02:51PM (#14213813)
Ex: "But the real reason for doing it is that it was a small amount of work that moved Objective-C forward, which allowed the compiler and language team to focus on Swift because Swift was a very large amount of work. " http://atp.fm/205-chris-lattner-interview-transcript/
Re:Swift architecture at Uber by Someone on Thursday April 27, 2017 @04:32AM (#14210042)
"So if you spawn 200 processes of Swift compilers"

I don't think that implies those processes all were spawned at the same time (or even that Apple's compilation process did the spawning. Given the effort they spent on their build processes, I do not rule out it is their tooling that does the spawning)

The thing is: all these processes are independent of each other. Uber likely has tons of files that are imported into many, many files. Each compilation process that imports such a file has to individually parse it.

I can see two ways around this: reusing intermediate results between processes and using fewer processes.

The first is like precompiled headers in C/C++. The second could mean having a queue of to be compiled files and a fixed number of compilation processes that pick up files from it.

Risk in both cases is that unintended state may leak between compilations. For example, compilation flags may be different for files f and g. Does that mean g can't use preprocessed module m generated while compiling f? Because of that, I would go for something like precompiled headers, because it makes it easier to reason about what information flows between compilations of f and g.

I don't know whether that would be sufficient, though. The biggest concern I have w.r.t. Swift is that all its cool features, combined, make it essentially impossible to build a fast compiler (Yes, languages such as C# have most of them and aren't that slow, but adding protocols and overflow detection (which you may want to optimize away a lot, even in unoptimized builds) may just tip the balance)

Re:Swift architecture at Uber by edgyswingset on Wednesday April 26, 2017 @11:41PM (#14209022)
The Swift compiler has some serious issues with the way it does type inference. Try compiling code with a line of basic arithmetic on a few numeric literals.
Re:Swift architecture at Uber by eridius on Wednesday April 26, 2017 @11:56PM (#14209067)
That has literally nothing to do with what the parent comment is talking about. And that's not naivety either. That's because the way type inference, operating overloading, and literals work together ends up being a combinatorial explosion.
Re:Swift architecture at Uber by bendecoste on Thursday April 27, 2017 @12:13AM (#14209124)
Kotlin has all of these features and build times don't (noticeably) suffer.
Re:Swift architecture at Uber by arcticbull on Thursday April 27, 2017 @01:00AM (#14209283)
As does Rust.
Re:Swift architecture at Uber by slavapestov on Thursday April 27, 2017 @01:01AM (#14209289)
From what I've heard Rust has serious build-time issues as well, mostly because of it's inability to perform separate compilation of generic code. Unlike Swift, Rust specializes everything at compile-time, similar to C++ templates.
Re:Swift architecture at Uber by steveklabnik on Thursday April 27, 2017 @02:09AM (#14209534)
That's not exactly true, while that is usually what Rust people write, you can choose to have stuff not monomorphized. I'm also not exactly sure what you mean by "separate" here.

As usual, compile times depend on what you're used to. It is something we're working on improving though; we want it to be very fast! We expect incremental recompilation to move out of nightly soonish; that will help quite a bit.

Re:Swift architecture at Uber by eridius on Thursday April 27, 2017 @01:40AM (#14209425)
No it doesn't. Rust doesn't have overloading, or operator overloading, or literal overloading. It literally has none of the pieces that cause the combinatorial explosion in Swift.
Re:Swift architecture at Uber by curun1r on Thursday April 27, 2017 @01:56AM (#14209478)
Rust does have operator overloading though the traits in the std::ops module. There are only a few (&&, ||, etc) that can't be overridden.
Re:Swift architecture at Uber by eridius on Thursday April 27, 2017 @02:07AM (#14209527)
Oh you're right about that. I was thinking of custom operators when I said no operator overloading, but of course that's not actually what we were discussing. Still, no literal overloading. Also, there are other differences between Rust and Swift's type systems that allow Rust to do type inference in a way that Swift cannot.