News for hackers. Stuff that matters.
Re:Towards a JavaScript Binary AST by Yoric on Friday August 18, 2017 @08:44AM (#15045584)
Well, it's more "compressed JavaScript" than "compiled JavaScript".
Re:Towards a JavaScript Binary AST by cabaalis on Friday August 18, 2017 @08:49AM (#15045608)
Something we badly need, I greatly appreciate the effort!
Re:Towards a JavaScript Binary AST by dmitriid on Friday August 18, 2017 @08:49AM (#15045606)
It's not. It's a binary representation of JS. Basically directly competing with WebAssembly.

What happens a year from now when WebAssembl gets support for GC and modules?

Re:Towards a JavaScript Binary AST by Yoric on Friday August 18, 2017 @08:54AM (#15045639)
Please don't tell me what the project is, I'm the tech lead :)
Re:Towards a JavaScript Binary AST by dmitriid on Friday August 18, 2017 @09:02AM (#15045697)
So why do you dismiss any and all arguments against it? Repeatedly.

You and your supporters consistently spread false information.

- WebAssembly does not support JS hence we need binary AST

WebAssembly is on track to support dynamic and GC-ed languages

- WebAssembly will be slower than binary AST

You've provided zero support for this statement. Meanwhile experiments with real apps (like Figma) show significant improvements with WebAssembly

- No one wants to ship bytecode

False

- it's hard to align browsers on bytecode

See WebAssembly

- you can't change bytecode once it's shipped

You can, eventually. Same argument can be applied to binary AST

- WebAssembly not supported by tools

Neither is binary AST. Meanwhile one of the goals of WebAssembly is tool support, readability etc.

There were other claims just as easily falsified, and yet dismissed out of hand.

So. What are you going to do with JS AST when WebAssembly gets support for JS?

Re:Towards a JavaScript Binary AST by tyingq on Friday August 18, 2017 @10:20AM (#15046396)
>WebAssembly is on track to support dynamic and GC-ed languages

It is, but there's several roadmap items that would need to be ticked off for it to run well. Aside from GC, you would probably need:

- Direct DOM access. The current way of interacting with the DOM from WASM is, at best, 10x slower.

- JIT underpinnings, like a polymorphic inline cache[1]

[1]https://github.com/WebAssembly/design/blob/master/FutureFeat...

It's not clear how far off those things might be.

Re:Towards a JavaScript Binary AST by batmansmk on Friday August 18, 2017 @01:55PM (#15048323)
Webassembly is pretty much a greenfield approach: you need to learn C, redo your whole codebase and still write JS code to load your wasm, interact with DOM and browser APIs, and leverage existing good enough JS libraries. This is the state of things as of 2017. It took 4 years to reach that point. If you want to bet everything on the state of wasm in 2-3 years from now, please be my guest. Meanwhile, I have a business to run.

The binary AST is a quick win for brownfield technology, with already a working version in an experimental Firefox build. It can be rolled quite fast. It improves right now the experience of many users on existing codebase the same way minification did. And improve the experience of current web assembly, as it is still largely dependent on the speed of JS. I'll take this feature any day.

Re:Towards a JavaScript Binary AST by Yoric on Friday August 18, 2017 @10:11AM (#15046322)
Let me clarify things one last time.

You are obviously very passionate and I believe that nothing I can write will convince you, so I will not pursue this conversation with you after this post.

For the same reason, my answers here are for people who have not followed the original HN thread in which we have already had most of this conversation.

I also believe that the best way for you to demonstrate that another approach is better than the JavaScript Binary AST would be for you to work on demonstrating your approach, rather than criticizing this ongoing work.

> - WebAssembly does not support JS hence we need binary AST

> WebAssembly is on track to support dynamic and GC-ed languages

If/when WebAssembly gains the ability to interact with non-trivial JS objects, JS libraries, etc. we will be able to compare the WebAssembly approach with the Binary AST approach. GC support and dynamic dispatch are prerequisites but are not nearly sufficient to allow interaction with non-trivial JS objects and libraries.

So let's meet again and rediscuss this if/when this happens.

> - WebAssembly will be slower than binary AST

> You've provided zero support for this statement. Meanwhile experiments with real apps (like Figma) show significant improvements with WebAssembly

If you recall, we are only talking about loading speed. I believe that there is no disagreement on execution speed: once WebAssembly implementations are sufficiently optimized, it is very likely that well-tuned WebAssembly code will almost always beat well-tuned JS code on execution.

The argument exposed on the blog is that:

- attempting to compile JavaScript to existing WebAssembly is very hard (hard enough that nobody does it to the best of my knowledge);

- the specifications of JavaScript are complex enough that every single object access, every single array access, every single operator, etc. is actually a very complex operation, which often translates to hundreds of low-level opcodes. Consequently, should such a compilation take place, I suspect that this would considerably increase the size of the file and the parsing duration;

- by opposition, we have actual (preliminary) numbers showing that compressing to JavaScript Binary AST improves both file size and parse time.

While I may of course be wrong, the only way to be sure would be to actually develop such a compiler. I have no intention of doing so, as I am already working on what I feel is a more realistic solution, but if you wish to do so, or if you wish to point me to a project already doing so, I would be interested.

You refer to the experiments by Figma. While Figma has very encouraging numbers, I seem to recall that Figma measured the speedup of switching from asm.js to WebAssembly. In other words, they were measuring speedups for starting native code, rather than JS code. Also a valid experiment, but definitely not the same target.

> - No one wants to ship bytecode

> False

Are we talking about the same blog entry?

What I claimed is that if we ended up with one-bytecode-per-browser or worse, one-bytecode-per-browser-version, the web would be much worse than it is. I stand by that claim.

> - it's hard to align browsers on bytecode

> See WebAssembly

I wrote the following things:

- as far as I know, browser vendors are not working together on standardizing a bytecode for the JavaScript language;

- coming up with a bytecode for a language that keeps changing is really hard;

- keeping the language-as-bytecode and the language-as-interpreted in sync is really hard;

- because of the last two points, I do not think that anybody will start working on standardizing a bytecode for the JavaScript language.

In case of ambiguity, let me mention that I am part of these "browser vendors". I can, of course, be wrong, but once again, let's reconvene if/when this happens.

> - you can't change bytecode once it's shipped

> You can, eventually. Same argument can be applied to binary AST

I claimed that no browser vendor seriously contemplates exposing their internal bytecode format because this would make the maintenance of their VM a disaster.

Indeed, there is an order of magnitude of difference between the difficulty of maintaining several bytecode-level interpreters that need to interact together in the same VM (hard) and maintaining several parsers for different syntaxes of the same language (much easier). If you know of examples of the former, I'd be interested in hearing about them. The latter, on the other hand, is pretty common.

Additionally, the JavaScript Binary AST is designed in such a manner that evolutions of the JavaScript language should not break existing parsers. I will post more about this in a future entry, so please bear with me until I have time to write it down.

> - WebAssembly not supported by tools

> Neither is binary AST. Meanwhile one of the goals of WebAssembly is tool support, readability etc.

I'm pretty sure I never said that.

> So. What are you going to do with JS AST when WebAssembly gets support for JS?

If/when that day arrives, I'll compare both approaches.

Re:Towards a JavaScript Binary AST by roywiggins on Friday August 18, 2017 @01:39PM (#15048176)
I may be mistaken, but I believe Adobe had to ship an Actionscript 2 runtime forever because they completely replaced it when they introduced AS3. They needed to keep the flash player backwards compatible, so the player had to have both runtimes. While it worked, it was a kludge.
Re:Towards a JavaScript Binary AST by dmitriid on Friday August 18, 2017 @10:43AM (#15046602)
> in which we have already had most of this conversation.

Interestingly enough, nowhere do you even mention these conversations in your attempts to push binary AST as hard as possible.

> Consequently, should such a compilation take place, I suspect that this would considerably increase the size of the file and the parsing duration;

Emphasis above is mine. However, it's presented (or was presented) by you as a fact.

> What I claimed is that if we ended up with one-bytecode-per-browser or worse, one-bytecode-per-browser-version, the web would be much worse than it is. I stand by that claim.

We are ending up with one wasm bytecode for every browser, aren't we?

> as far as I know, browser vendors are not working together on standardizing a bytecode for the JavaScript language;

Because they are working together on standardizing bytecode for the web in general, aren't they?

> coming up with a bytecode for a language that keeps changing is really hard

So you're trying to come up with a binary AST for a language that keeps changing :-\

> keeping the language-as-bytecode and the language-as-interpreted in sync is really hard

What's the point of WebAssembly then?

> I do not think that anybody will start working on standardizing a bytecode for the JavaScript language.

Because that's not really required, is it? This is a non-goal, and never was the goal. The goal is to create an interoperable standardized bytecode (akin to JVM's bytecode or .Net's MSIL), not a "standardized bytecode for Javascript". For some reason you don't even want to mention this.

> I claimed that no browser vendor seriously contemplates exposing their internal bytecode format

They don't need to.

> Additionally, the JavaScript Binary AST is designed in such a manner that evolutions of the JavaScript language should not break existing parsers.

I shudder to think how rapidly changing languages like Scala, ClojureScript etc. can ever survive on the JVM. The horror!

> If/when that day arrives, I'll compare both approaches.

Basically you will end up with two representations of JS: a binary AST and a version that compiles to WASM. Oh joy. Wasn't this something you wanted to avoid?

Re:Towards a JavaScript Binary AST by orf on Friday August 18, 2017 @01:12PM (#15047958)
Regardless of all your points, the fact is that WASM isn't ready for this now, and doesn't appear that it will be for some time.

Combined with the fact, as mentioned above, that DOM access is significantly slower means that WASM isn't a suitable candidate. This is something you forgot to mention or take into account in your comment, for some reason.

Yes this does seem to overlap a bit with wasm as you have noted, but saying "well we could get this optimization but we need to wait several years until this highy complex other spec is completely finished" doesn't seem as good.

Why not do this, and use wasm when it's available? Why can't you have both?

Re:Towards a JavaScript Binary AST by dmitriid on Saturday August 19, 2017 @12:04PM (#15054088)
> Regardless of all your points, the fact is that WASM isn't ready for this now

As opposed to Binary AST which as available now? :)

> Combined with the fact, as mentioned above, that DOM access is significantly slower

Given that DOM access for WASM currently happens via weird interop through JavaScript (if I'm not mistaken) how is this a fact?

> "well we could get this optimization but we need to wait several years until this highy complex other spec is completely finished" doesn't seem as good.

No. My point is: "Let's for once make a thing that is properly implemented, and not do a right here right now short-sighted short-range solution"

That's basically how TC39 committee operates these days: if something is too difficult to spec/design properly, all work is stopped and half-baked solutions are thrown in "because we need something right now".

> Why not do this, and use wasm when it's available? Why can't you have both?

Because this means:

- spreading the resources too thin. There is a limited amount of people who can do this work

- doing much of the work twice

- JS VM implementors will have to support text-based parsing (for older browsers), new binary AST and WASM

Re:Towards a JavaScript Binary AST by orf on Saturday August 19, 2017 @12:21PM (#15054182)
> how is this a fact?

... because DOM access is significantly (10x) slower? That alone rules out your approach right now, regardless of the other points raised.

> Because this means: ...

When you understand how weak these arguments are you will understand the negative reaction to your comments on this issue.

You're clearly very passionate about this issue but you don't seem to be assessing the trade offs. Having something right now and iterating on it is better than waiting an indeterminate amount of time for a possible future solution involving an overlapping but otherwise unrelated specification that has not been fully implemented by anyone to a satisfactory point, and one with very complex technical issues blocking its suitability.

Sure, it would be nice to use WASM for this, but it is in no way needed at all. Given the status of WASM and the technical issues present with using it in this way it is odd to champion it to such a degree.

It seems your entire arguments boil down to "WASM may be OK to use at some point in the future, stop working on this and wait!". I, and I'm assuming others, don't see this as a very convincing point.

If I may, I'd offer some advice: stop taking this issue to heart. Time will tell if you're right, and making borderline insulting comments to the technical lead of the project in an attempt to push your position doesn't help anyone.

The world is heating up and species are dying, there are much better causes to take to heart.

Re:Towards a JavaScript Binary AST by MadcapJake on Friday August 18, 2017 @09:38AM (#15045973)
How can you say it's "compressed" over "compiled" when you are actually parsing it into an AST and then (iiuc) converting that to binary? That's exactly what compilers do. You are in fact going to a new source format (whatever syntax/semantics your binary AST is encoded with) so you really are compiling.

To be fair, these two concepts are similar and I may be totally misunderstanding what this project is about. In the spirit of fairness, let me test my understanding. You are saying wasm bytecode is one step too early and a true "machine code" format would be better able to improve performance (especially startup time). I'm not following wasm development, but from comments here I am gathering that wasm is too level and you want something that works on V8. Is that what this project is about?

On a side note, it's truly a testament to human nature that the minute we get close to standardizing on something (wasm), someone's gotta step up with another approach.

Re:Towards a JavaScript Binary AST by Yoric on Friday August 18, 2017 @10:24AM (#15046430)
> How can you say it's "compressed" over "compiled" when you are actually parsing it into an AST and then (iiuc) converting that to binary? That's exactly what compilers do. You are in fact going to a new source format (whatever syntax/semantics your binary AST is encoded with) so you really are compiling.

I am not sure but there may be a misunderstanding on the word "binary". While the word "binary" is often used to mean "native", this is not the case here. Here, "binary" simply means "not text", just as for instance images or zipped files are binary.

A compiler typically goes from a high-level language to a lower-level language, losing data. I prefer calling this a compression mechanism, insofar as you can decompress without loss (well, minus layout and possibly comments). Think of it as the PNG of JS: yes, you need to read the source code/image before you can compress it, but the output is still the same source code/image, just in a different format.

> You are saying wasm bytecode is one step too early and a true "machine code" format would be better able to improve performance (especially startup time). I'm not following wasm development, but from comments here I am gathering that wasm is too level and you want something that works on V8. Is that what this project is about?

No native code involved in this proposal. Wasm is about native code. JS BinAST is about compressing your everyday JS code. As someone pointed out in a comment, this could happen transparently, as a module of your HTTP server.

> On a side note, it's truly a testament to human nature that the minute we get close to standardizing on something (wasm), someone's gotta step up with another approach.

Well, we're trying to solve a different problem :)

Re:Towards a JavaScript Binary AST by dblohm7 on Friday August 18, 2017 @01:47PM (#15048239)
Posting this in the hope that it might help some people grok what they are actually doing:

When I first discovered what Yoric and syg were doing, the first thing that I thought of was old-school Visual Basic. IIRC when you saved your source code from the VB IDE, the saved file was not text: it was a binary AST.

When you reopened the file in the VB6 IDE, the code was restored to text exactly the way that you had originally written it.

Re:Towards a JavaScript Binary AST by Yoric on Friday August 18, 2017 @02:04PM (#15048403)
Interesting. Do you know of any technical documentation on the topic?
Re:Towards a JavaScript Binary AST by meredydd on Friday August 18, 2017 @02:44PM (#15048703)
The parent might have been thinking of QuickBasic, which saved programs as byte-code, along with formatting information to turn it back into text: http://www.qb64.net/wiki/index.php/Tokenized_Code

VB6 projects were actually a textual format - even the widget layout: https://msdn.microsoft.com/en-us/library/aa241723(v=vs.60).a...

As someone who runs a WYSIWYG app builder startup (https://anvil.works), I can attest that this is a Really Good Idea for debugging your IDE.

Re:Towards a JavaScript Binary AST by iainmerrick on Friday August 18, 2017 @03:41PM (#15049132)
BBC Basic did this too -- keywords were stored as single bytes (using the 128-255 values unused by ASCII). Apart from that the program code wasn't compiled, it just was interpreted directly. Very smart design when everything had to fit in 32KB of RAM.
Re:Towards a JavaScript Binary AST by MadcapJake on Friday August 18, 2017 @02:09PM (#15048433)
> Here, "binary" simply means "not text", just as for instance images or zipped files are binary.

If it's not text, then what is it? I'm not sure "not text" is a good definition of the word "binary".

> A compiler typically goes from a high-level language to a lower-level language, losing data.

I don't agree, I don't think there is any loss in data, the compiled-to representation should cover everything you wanted to do (I suppose not counting tree-shaking or comment removal).

> I prefer calling this a compression mechanism, insofar as you can decompress without loss (well, minus layout and possibly comments).

Ahh, so you mean without losing the original textual representation of the source file.

> Wasm is about native code

Here you are making claims about their project that are just not the whole picture. Here's the one-line vision from their homepage[1]:

> WebAssembly or wasm is a new portable, size- and load-time-efficient format suitable for compilation to the web.

With that description in mind, how do you see BinAST as different?

> Well, we're trying to solve a different problem :)

I think you might be misunderstanding what wasm is intended for. Here's a blurb from the wasm docs that is pertinent:

> A JavaScript API is provided which allows JavaScript to compile WebAssembly modules, perform limited reflection on compiled modules, store and retrieve compiled modules from offline storage, instantiate compiled modules with JavaScript imports, call the exported functions of instantiated modules, alias the exported memory of instantiated modules, etc.

The main difference I can gather is that you are intending BinAST to allow better reflection on compiled modules than wasm intends to support.

Here's another excerpt from their docs (and others have mentioned this elsewhere):

> Once GC is supported, WebAssembly code would be able to reference and access JavaScript, DOM, and general WebIDL-defined objects.

[1]: http://webassembly.org/

[META: Wow, I thought downvotes were for negative or offtopic comments]

Re:Towards a JavaScript Binary AST by MadcapJake on Friday August 18, 2017 @02:23PM (#15048533)
Ok I am understanding the distinction now.

I ran a google search for "js to wasm" and found a ticket on the webassembly github that explained it all: https://github.com/WebAssembly/design/issues/219

Re:Towards a JavaScript Binary AST by Yoric on Friday August 18, 2017 @02:29PM (#15048593)
Thanks for the link, it will certainly prove useful in the future.
Re:Towards a JavaScript Binary AST by pharrington on Friday August 18, 2017 @07:49PM (#15050637)
Text means a sequence of characters conforming to some character encoding. Yoric's binary AST is not a sequence of characters conforming to a character encoding.

Compilation maps a program in a higher level language to a program in a lower level language. The map is not required to be one-to-one: the colloquial term for this is "lossy."

Re:Towards a JavaScript Binary AST by nfriedly on Friday August 18, 2017 @09:57AM (#15046191)
To quote the original post:

> If you prefer, this Binary AST representation is a form of source compression, designed specifically for JavaScript, and optimized to improve parsing speed.