News for hackers. Stuff that matters.
Re:Speedy Web Compiler – A Rust port of Babel and Closure Co... by ajross on Sunday December 23, 2018 @10:47AM (#18746671)
Seems like transcompilation is an area where you really DON'T want be overfocusing on performance. Correctness is king here. I mean, compiler bugs are hard. Really hard. Deep voodoo hard. No one who's ever dealt with a significant code generation bug on a large project ever wants to repeat that experience.

That's not to say that babel isn't too slow or that this project isn't correct, just that it would scare me to try to work with it. Does babel have a regression suite? Does this pass it?

Re:Speedy Web Compiler – A Rust port of Babel and Closure Co... by peterhunt on Sunday December 23, 2018 @10:49AM (#18746685)
Js packing performance (including compilation) is probably the slowest part of the iteration cycle for frontend devs today. When combined with Babel’s and ecmas extensive test suites I think something like this could be wildly successful.
Re:Speedy Web Compiler – A Rust port of Babel and Closure Co... by rcavezza on Sunday December 23, 2018 @10:59AM (#18746741)
Can you elaborate more on this? I've worked with very large codebases in React and Angular and have not seen this issue. I'm guessing it's either because all of my applications were able to hot-reload or we're not thinking of the same thing.
Re:Speedy Web Compiler – A Rust port of Babel and Closure Co... by the_mitsuhiko on Sunday December 23, 2018 @11:06AM (#18746793)
I don’t know if we’re doing something wrong but initial startup time on sentry’s webpack setup is something like 30 seconds and iterative changes around 5 seconds. Still beats our rust or python iteration times though :)
Re:Speedy Web Compiler – A Rust port of Babel and Closure Co... by orf on Sunday December 23, 2018 @04:17PM (#18748383)
Have you looked into why the Sentry Django app takes so long to boot? Are you doing queries on app init or something?
Re:Speedy Web Compiler – A Rust port of Babel and Closure Co... by peterhunt on Sunday December 23, 2018 @11:07AM (#18746803)
a lot of codebases don't support hot reload, but even the ones that do often require multi-second incremental compile times during hot reload if they're big enough [1].

[1] current project is very slow w/ 250kloc (2.5mm kloc w/ dependencies). even seen it be slow on my last project which was only 40kloc.

Re:Speedy Web Compiler – A Rust port of Babel and Closure Co... by mort96 on Sunday December 23, 2018 @11:34AM (#18746948)
Just using tsc to compile a hello world with 7 tokens takes over a second in my experience. Sure, most of that is initialization, but compare that to the C and C++ world, where GCC spends tens of milliseconds at most compiling hello world.

Hot reloading _helps_, but it's definitely an issue that javascript tooling spends seconds on tasks other languages' tooling spends tens or hundreds of milliseconds on. (The issue isn't just limited to tsc either; just starting the build system can take a decent amount of time with webpack, while make or cmake starts in milliseconds.)

Re:Speedy Web Compiler – A Rust port of Babel and Closure Co... by hzoo on Sunday December 23, 2018 @11:26AM (#18746905)
If you can use test262's test suite you should be good, it looks like they are referring to Babel's tests for some of the features? example https://github.com/swc-project/swc/pull/86, I just checked really quick though.

In terms of perf, I'm sure rust will be much faster than JS. And this project's main goal is to be a faster version than Babel. I would suggest the perf test would be different though because it's testing code I would consider not to be representative of what Babel actually runs on. Example with the transform test: https://github.com/swc-project/swc/blob/222bdc191fcab7714319... it's a very small file with minimal transforms even necessary at all. And the parser test is for all minified js files, none of which are written in ES6+ https://github.com/swc-project/swc/blob/master/ecmascript/pa.... Currently it's not really testing ES6+, just how at a baseline it's much faster to process a ES5 file, so it's possible that it might be even faster.

edit: I will note that Babel does a lot more than what this project is doing currently (plugins, other proposals, ts/flow, sourcemaps, etc) but this is more about what could be so if people are willing to more on this effort in the long run (with funding/support/people) it could be super useful! I'm curious about how that will work when our project has such a struggle with contributors and Babel itself is written in JS. (I work on Babel :D)

Re:Speedy Web Compiler – A Rust port of Babel and Closure Co... by the_mitsuhiko on Sunday December 23, 2018 @11:02AM (#18746754)
It uses the tc39 testsuite for verification: https://github.com/tc39/test262
Re:Speedy Web Compiler – A Rust port of Babel and Closure Co... by moron4hire on Sunday December 23, 2018 @12:54PM (#18747337)
I'm sorry, but I'm kind of tired of the idea that some problems are "deep voodoo hard". It casts it in an inaccessible light, creating a culture that new folks internalize, guiding then away from ever even trying.

Unless we're talking about "understanding neural network internals", which is a big open problem, there just really aren't "black magic" problems inherent to particular subjects. It's almost always poor spec definition or poor code design, if there are any hand-waivey bits to be found.

If anything, compiler design (along with other subjects like text editor development) is a very deeply studied, very well understood, very well documented field. It should be easier than any other fields less central to computer science and software development, just on account of not being so thoroughly studied.

EDIT: fixed autocorrect problems

Re:Speedy Web Compiler – A Rust port of Babel and Closure Co... by chrisseaton on Sunday December 23, 2018 @01:34PM (#18747552)
Many people working on compilers will have a PhD in the subject. It's an area where it definitely takes many years to get up to speed. As you said yourself it's deeply studied, and studying all that work takes ages.

Compilers are also definitely not well documented. I think they're rather notoriously undocumented with much of our knowledge of the subject passed on orally and existing only in the knowledge of people working in the field.

For example a couple of years ago I went looking for details of a very common compiler technique, called safepoints, used in many major industrial compilers today, and found that there was essentially no written information on them at all. That's pretty extraordinary.

Re:Speedy Web Compiler – A Rust port of Babel and Closure Co... by hyperpape on Sunday December 23, 2018 @01:58PM (#18747679)
Two thoughts:

1. Safepoints are kind of off-topic, because they're part of the interplay between a complex optimizing compiler and the garbage collector/runtime. Babel and its ilk are source-source compilers targeting a language that doesn't have manual memory management. To my mind, that makes things much simpler.

2. How long ago were you searching for safepoint info? I've collected several articles and talks relating to safepoints. I don't know of anything that discusses safepoints in a way that you could implement them from scratch just based on the discussion, but they're tied to a complex part of the runtime, so any example/tutorial code would probably be parochial to a particular VM.

Re:Speedy Web Compiler – A Rust port of Babel and Closure Co... by chrisseaton on Sunday December 23, 2018 @02:04PM (#18747721)
> I don't know of anything that discusses safepoints in a way that you could implement them from scratch just based on the discussion

The best discussion of them at the moment is this one - and as you'll notice they're essentially having to do analysis of what is already out there because it just wasn't written down anywhere at the time. They don't have much of a 'related work' section or a bibliography because there wasn't much to refer to.

https://dl.acm.org/citation.cfm?id=2754169.2754187

And implementing them from scratch is what I wanted to know how to do. For a lot of compiler topics the only way to learn is to sit down with an existing expert and physically ask them. Compared to other CS fields that’s really bad.

Re:Speedy Web Compiler – A Rust port of Babel and Closure Co... by pbiggar on Sunday December 23, 2018 @06:40PM (#18749196)
> Many people working on compilers will have a PhD in the subject.

There is some truth to this, but as someone with a PhD in compilers, let me say that this area is very accessible to people without PhDs. The first time I met @dannybee, I asked him where he went to grad school. Turns out he went to law school. Didn't stop him writing the GCC alias analysis framework though.

Re:Speedy Web Compiler – A Rust port of Babel and Closure Co... by pjmlp on Monday December 24, 2018 @02:12AM (#18750885)
How much time did you spent in CS literature for compilers?

One of my focus areas during my degree was system programming, including compilers and language design.

Already in the late 90s, there were several good books about compilers.

Re:Speedy Web Compiler – A Rust port of Babel and Closure Co... by chrisseaton on Monday December 24, 2018 @07:17AM (#18751804)
> Already in the late 90s, there were several good books about compilers.

There are tons of introductory texts, a few intermediate texts, a couple of very narrow advanced texts, and then it just runs out. A great deal of our knowledge is undocumented.

Re:Speedy Web Compiler – A Rust port of Babel and Closure Co... by tedunangst on Sunday December 23, 2018 @01:03PM (#18747376)
He didn't say compilers were voodoo. Diagnosing subtle bugs in a complex code generator transformer is voodoo. If you haven't written the compiler, but are just an end user, I'd say that's a fair assessment. It's a lot of work to identify what went wrong.
Re:Speedy Web Compiler – A Rust port of Babel and Closure Co... by polskibus on Sunday December 23, 2018 @11:00AM (#18746743)
The slower the build the more developer's focus fades away whether he wants it or not. I'd say performance of build tools is at least as important as their correctness :) compiler bug can be fixed but developers' focus not very much.
Re:Speedy Web Compiler – A Rust port of Babel and Closure Co... by empath75 on Sunday December 23, 2018 @10:52AM (#18746705)
If you want bug free code, rust is as good a choice as any to write it in.
Re:Speedy Web Compiler – A Rust port of Babel and Closure Co... by ajross on Sunday December 23, 2018 @11:19AM (#18746866)
Not when compared to mature software it isn't, no. The comparison isn't to "Javascript" as an implementation language for a transcompiler, it's to "Babel" as an actual transcompiler in wildly active use to generate production code everywhere in the world.
Re:Speedy Web Compiler – A Rust port of Babel and Closure Co... by hyperpape on Sunday December 23, 2018 @11:53AM (#18747061)
This is true of every upstart project, though. LLVM was a risky project at one point. People with high risk tolerance will pave the way, and the rest of us will join in if/when the project seems mature.
Re:Speedy Web Compiler – A Rust port of Babel and Closure Co... by cgag on Sunday December 23, 2018 @01:38PM (#18747574)
Though when that mature codebase and the spec come with solid test suites you can run against it seems a lot less concerning.
Re:Speedy Web Compiler – A Rust port of Babel and Closure Co... by nkcmr on Sunday December 23, 2018 @10:59AM (#18746742)
What part of rust prevents the writing of bugs? I understand it helps you avoid unsafe memory operations, but to say it gives you “bug free” code seems like an falsehood.
Re:Speedy Web Compiler – A Rust port of Babel and Closure Co... by bluejekyll on Sunday December 23, 2018 @11:13AM (#18746844)
Rust doesn’t implicitly create bug free code. I’ve created lots of bugs in Rust.

But there are classes of bugs that the language eliminates. Thread unsafe dataraces (not important in JS), unhandled exceptions, with no GC it helps reduce memory usage and make it more consistent, the type system allows you to build complex state machines easily and correctly, the type system itself can allow you to express states that should never occur as compilation failures, the compiler protects against concurrent updates in iterators and other situations.

The list goes on, but these are the things that make it such a powerful language and allow me to focus on logic and not runtime bugs. Oh, and it’s really fast which is a nice bonus, generally as fast as C.

Re:Speedy Web Compiler – A Rust port of Babel and Closure Co... by wwright on Sunday December 23, 2018 @12:06PM (#18747104)
To add: safe Rust prevents all data races, not just thread based ones (this is mostly relevant around iterators and other concurrency-without-parallelism structures, but is relevant more generally as well)
Re:Speedy Web Compiler – A Rust port of Babel and Closure Co... by steveklabnik on Sunday December 23, 2018 @11:03AM (#18746772)
Rust can not guarantee correctness, but it is a focus in some sense. Strong static typing, robust error handling... you can encode a lot of invariants in the type system. Especially compared to something dynamically typed, like JavaScript. It’s not a proof assistant, but every bit helps!
Re:Speedy Web Compiler – A Rust port of Babel and Closure Co... by nkcmr on Sunday December 23, 2018 @11:11AM (#18746828)
Thanks for correcting me there. Rust _does_ have a more than the borrow checker that are helping the programmer avoid common dynamic language issues (and rust is awesome).

I just know that even if the compiler of the language I am writing is very smart and can catch a lot of my silly mistakes, it would behoove me to think that I could write "bug free" code as a result. I would still write my code defensively, even in rust.

Re:Speedy Web Compiler – A Rust port of Babel and Closure Co... by steveklabnik on Sunday December 23, 2018 @11:16AM (#18746856)
Absolutely. I’ve written many bugs in Rust. But on the whole, my first attempts have less bugs overall. YMMV, of course.
Re:Speedy Web Compiler – A Rust port of Babel and Closure Co... by leshow on Sunday December 23, 2018 @03:50PM (#18748233)
Should be noted it usually takes a bit longer to get to that first working version in Rust. It's debatable whether in the long run it's faster to develop with.

I would argue it is much faster to get to a final, 'correct' working version because you spend less time debugging. Time is also drastically cut down the more familiar you are with the language, but still, I'd love to see some data on dev time between languages with robust type systems vs dynamic.

Re:Speedy Web Compiler – A Rust port of Babel and Closure Co... by KMag on Sunday December 23, 2018 @08:25PM (#18749629)
Do you mean fewer bugs, or less severe bugs (or some other non-count metric other than severity)?
Re:Speedy Web Compiler – A Rust port of Babel and Closure Co... by chrismorgan on Monday December 24, 2018 @12:17AM (#18750504)
I expect my experience is much the same as Steve’s on this point.

Mostly I would say fewer bugs; the bugs that are avoided vary in severity. Many, probably most, are the severe but obvious bugs where you would have hit the first time you tried running the code in another language, but which are instead now caught by the compiler—that’s why “time to first run” is not a reasonable metric for comparing writing code in such languages, but “time to first meaningfully successful run”! But it’s also common for Rust to catch and prevent far more insidious bugs; especially, in my experience, in two areas: firstly, with respect to ownership—things like passing data structures around and improperly mutating them where you should have cloned it instead, which is structurally impossible in Rust (disregarding things like Rc<RwLock<T>>, which make it possible, but make the intent very much clearer); and secondly, where enums (tagged unions) can be used, due to the greater and more precise expressivity.

I have much greater confidence that my first pass at a piece of code will work correctly in Rust, once it compiles, than I do in JavaScript—and I’ve been working in JavaScript a lot more than Rust in the last couple of years, yet it’s still true.

Re:Speedy Web Compiler – A Rust port of Babel and Closure Co... by steveklabnik on Monday December 24, 2018 @09:45AM (#18752324)
Yes, this is exactly what I mean.
Re:Speedy Web Compiler – A Rust port of Babel and Closure Co... by rubber_duck on Sunday December 23, 2018 @11:10AM (#18746824)
>What part of rust prevents the writing of bugs

Expressive type system, strong and static - JS is weak and dynamic. This is not limited to Rust - but if you're aiming for correct code pretty much anything after C/C++ and maybe PHP is better than JS and it's million and one way to write unexpected bugs and 10.000 ways to do the same basic thing (with an NPM package no less) of which each dev/library chooses it's own path for cool factor.

Who would even argue that JS is better for writing correct code over a language that's obsessive about security and correctness. Last time I checked Rust wouldn't even upcast ints FFS, meanwhile in JS you can't even be sure what "this" refers to in any given context and 0 == '0' == [0].

Re:Speedy Web Compiler – A Rust port of Babel and Closure Co... by pythux on Sunday December 23, 2018 @11:34AM (#18746947)
Babel makes extensive use of Flow. So I guess that counts as expressive static typing.
Re:Speedy Web Compiler – A Rust port of Babel and Closure Co... by the_mitsuhiko on Sunday December 23, 2018 @11:03AM (#18746765)
Doesn’t “as good as any” just mean “not worse than any other language”?
Re:Speedy Web Compiler – A Rust port of Babel and Closure Co... by pasta1212 on Monday December 24, 2018 @08:43AM (#18752074)
Yes, but why would you say it in the negative form? Both forms mean it’s a tie or better.
Re:Speedy Web Compiler – A Rust port of Babel and Closure Co... by eiurafhlfie on Sunday December 23, 2018 @11:11AM (#18746831)
Tagged union types (called `enum` in Rust), recursion, and pattern matching are great for tree data structures, such as ASTs, and help catch many bugs. This is the reason why so many compilers, typecheckers, and interpreters are written in languages like OCaml, F#, Haskell, and Rust.
Re:Speedy Web Compiler – A Rust port of Babel and Closure Co... by snaky on Sunday December 23, 2018 @11:17AM (#18746859)
> OCaml, F#, Haskell, and Rust

If you change Rust to ML, the phrase would be way more natural.

Re:Speedy Web Compiler – A Rust port of Babel and Closure Co... by n-gate on Sunday December 23, 2018 @11:40AM (#18746991)
How many non-toy non-self compilers and interpreters are written in Rust?
Re:Speedy Web Compiler – A Rust port of Babel and Closure Co... by steveklabnik on Sunday December 23, 2018 @11:48AM (#18747030)
There’s at least a few; cranelift is powering stuff in production at Fastly. Facebook is writing a MIR interpreter to do static analysis, don’t think that’s production yet though.
Re:Speedy Web Compiler – A Rust port of Babel and Closure Co... by peterhunt on Sunday December 23, 2018 @11:04AM (#18746781)
certainly won't guarantee bug free code, but exhaustive pattern matching really helps when you're building compilers (as it does in other languages that have this feature).
Re:Speedy Web Compiler – A Rust port of Babel and Closure Co... by richardwhiuk on Sunday December 23, 2018 @11:03AM (#18746763)
You can mark values (e.g. Error conditions) as Must Use. This, along with structured enumerated types, means that you can't forget to handle error conditions.

Other bugs such as the goto fail bug are eliminated by dead code spotting and consistent formatting.

You can write bugs in Rust but they tend to be business logic bugs, or incorrect explicit decisions.

Re:Speedy Web Compiler – A Rust port of Babel and Closure Co... by empath75 on Sunday December 23, 2018 @11:03AM (#18746773)
It doesn’t. I didn’t say it did.
Re:Speedy Web Compiler – A Rust port of Babel and Closure Co... by jasonkester on Sunday December 23, 2018 @11:00AM (#18746745)
But you're talking about a step that javascript developers want to inflict on themselves every time they change anything. If the time to do that is not exactly zero, it will kill productivity [1].

[1] Naturally excluding the 18 hours of productivity that will have been killed getting said unnecessary javascript->javascript build system set up and running in the first place.

Re:Speedy Web Compiler – A Rust port of Babel and Closure Co... by swsieber on Sunday December 23, 2018 @12:34PM (#18747245)
It's a trade off. People aren't stupid or masochistic for making it.