You are speaking my language! I would love to see a prototype of this. I've been thinking about it since I saw React Native.

I wonder if there's even value in something like an asm.js-compiled layout system, which absolutely positions elements. One of the insights of React Native is that to build apps, we really only need a small subset of the web's layout algorithms. Only flexbox, really. So if we recognize this subset, and remove everything else, there main thread can focus on really smooth performance.

React Native even performs the flexbox layout on a 3rd thread, and tells the main thread where to absolutely position stuff.

Obviously we want to re-use as much of HTML as we can, but there might be some crazy ways to remove stuff from the main thread.

EDIT: It should be noted that Mercury, a React-inspired library (https://github.com/Raynos/mercury), has examples of running in a web worker. It is pretty cool, but I think it needs to be fleshed out more.

We explored this at a previous company I worked at. The conclusion at the time was that the message passing model was not yet efficient enough at the time because of serialization/deserialization which nullified any performance gains. Transferrable objects at the time were really new. If the support for shared memory has improved, it might now make sense.

The biggest problems with performance are:

- GC pauses

- triggering reflow

- marshalling data across interfaces (serialization/deserialization of native types into strings and back again).

That last one is an absolute killer when it comes to applying 3D matrix transforms. Basically, you're taking an array of floats, serializing them into a string to do an element.style property assignment, so that the browser can then take that stringified representation and convert it back into an array of floats that it can apply. Huge perf hit here.

I haven't found serialization to be anywhere near the bottleneck. How long does it take to serialize/deserialize a 16 element array? I could see how this would be an issue if you had hundreds of them per frame. But for most purposes an entire 16 element array is overkill and you could probably come up with a more compressed description of the transforms (scale, rotate, translate) and consider adding a pooling layer.

People attribute pauses to GC, when it's sometimes just the browser container environment acting up and there's nothing we can do about it - so it's easy to write it off as GC. Not sure about today, but 1.5 years ago (i = new Image(), i.src="http...") could block the JS thread for as much as 19ms. In a larger app, it's easy to write that off as GC, but (at least on iOS) the performance tooling is really lacking and it's tough to find the culprit.

Reflowing is a big deal, I agree.

Basically if you want to prevent reflowing you end up making everything a texture and that leads you to making many many CSS transform changes on every frame. 60fps is easy to hit on a modern desktop, but mobile performance requires more effort to achieve an effect that feels natural. The problem with the marshalling is that it is comparatively expensive and a blocking operation, which is not conducive to buttery smooth graphics performance on all platforms.

Blowing out texture memory accidentally was also a big deal when I was measuring this stuff.

You can now transfer typed array data using the newest postMessage API. http://updates.html5rocks.com/2011/12/Transferable-Objects-L...

Me me! I'll be exploring this soon and I'm sure I'll reach out for help before I get too far. Could do a lot of stuff off thread, from data to layout calcs. My profile should have some info related to the hybrid stuff.

Very cool! I'll keep an eye on your work!

Me too.

I'd love to chat more about this with both of you, this seems like a direction definitely worth exploring.

Maybe we could set up a github project for it? Or talk in an issue on your existing repo, nwienert?

I'd like to see a github repo. You can hit me up on twitter, info in my profile. I think even just finding the right library to make using webworkers + npm + webpack would be great to start, because honestly its all the other stuff that slows down the thread. React can run on the main thread for the most part.

Some reading:

https://news.ycombinator.com/item?id=6982485 http://stackoverflow.com/questions/18056922/is-there-a-way-t...

Edit: webpack + webwoker: https://github.com/webpack/webpack/tree/master/examples/web-...

Cool, I set up a repo and started a wiki on it,

https://github.com/kripken/worker-ui/wiki

Not familiar with webpack, will take a look.

If you're interested in doing it with React, here's a video explaining how to do a custom backend: https://www.youtube.com/watch?v=eNC0mRYGWgc

You basically need to change BackendIDOperations and ReactEventListener to send/receive messages vs talking to the real APIs. It's already set up in the codebase to do this (because it actually worked at one time).

Webpack, per usual, already thought of this and has an answer for you :) https://github.com/webpack/worker-loader

Sweet. I'll watch out for wherever you guys end up.

Might be worth reading one of the react native devs about why animations might still be hard: http://jlongster.com/First-Impressions-using-React-Native#co...

This idea is also why I hooked up the css-layout project (https://github.com/facebook/css-layout) to a live demo to play around with: http://layout.jlongster.com/

Yes, in browsers, JS animations are still hard (I might even say impossible to do well for large apps with a lot going on). One anecdote: I spent a month trying to build a 60fps scrolling list view in JS from scratch (JS driving all the animations). No matter how hard I tried, I couldn't make it work on the web. I believe it was image decoding interfering but it's difficult to say why. Then I took that same example, which by that point was optimized, and ported it to React Native in a few hours(where we have off-thread image decoding and timers I can trust) and the example was like butter.

Now of course, I had hand optimized the performance, but my point is that on the web, there was nothing I could do, and with React Native, there was something I could do to make animations smooth. This is not to say you should be doing animations in JS, but if you want highly customizable interactions that can be immediately stopped by a touch processed in JS, you should try it out and see how it feels. For things that are "fire and forget", or platform specific, you should try using CoreAnimation/keyframes etc which protects the animation from your business logic.

Interestingly, this comment, while probably being of the most technical detail / value in the thread, was downvoted by someone :)

what's wrong with -webkit-overflow-scrolling: touch? doing scrolling in js is so last year.

You can't build the equivalent of UITableView (which is a very important component in almost any iOS app) without frame-by-frame scroll events.

How so? Which feature of the UITableView can not be replicated in JS? I'm struggle to think of one. Swipe left/right to remove rows in a table can be done with touch events.

What am I missing?

I think you're not being specific enough. I am claiming they can perfectly be recreated in JS, just not within a mobile browser at the moment.

I do mean within a mobile browser. We have a pretty large app inside Cordova, and I've yet to get stuck where there's something I can't recreate. Would be interesting to see an example where a mobile browser falls short.

Right, but then you're susceptible to animation stutter and you have to reinvent the look-and-feel of the platform, which is extremely difficult.

Exactly. Customizing exactly how a header bar "sticks" to the top when it hits a certain point, cannot be done with any built in browser scroller. The same is true of any animation that should happen in sync with the scrolling itself (think like a parallax fade/scale animation etc).

What most apps need can be done by listening to touch events instead of scroll events. Eg hiding the top bar when user started scrolling down and so on.

I can't even think of an app that has a parallax effect. If your app is required to have complex animations then web tech is not the right tech choice in general imo

Also worth looking at some ideas we have for doing cross-thread component hierarchies in React:

https://github.com/reactjs/react-future/tree/master/05%20-%2...

oh nice, thanks for mentioning this.

I am not a maintainer and I just tried it briefly so I am not 100% sure of the allegations they make on their front-page, but it seems like http://gridstylesheets.org are using Workers to run their layout algorithm (Cassowary Constraint Solver) and absolute-position elements (using 3D transforms).

EDIT: Searching for "Worker" right now didn't bring any mention, but it seems it is a very recent edit (cf https://github.com/gss/gss.github.io/commit/d6a931e3629d4b5d...).

> I think it's a very strong point that moving script code off the main thread can help achieve smooth UIs. No more GC pauses, no more slowdowns if the JS engine hits a snag, etc.

BeOS took UI multithreading farther than any other environment I've ever used. And it paid off: I still think it might be the case that BeOS on circa 2000 hardware was more consistently responsive than any environment I have used before or since.

More technical detail about BeOS's multithreaded UI design: http://arstechnica.com/civis/viewtopic.php?p=17348543&sid=e2...

> I think it's a very strong point that moving script code off the main thread can help achieve smooth UIs.

What, you mean like we do in Servo? https://github.com/servo/servo :-)

Our main thread (if you can call it that) just does compositing and handles dispatching UI events. Script, layout, resource loading, and the rest run concurrently in background threads.

That's amazing. I hadn't even though about Servo in light of all of this.

In current browsers, JavaScript runs in the same thread as the UI, right? In Servo, how do you do interaction with the DOM like `clientHeight`? It just pauses the entire JS engine while it goes and talks to the UI thread?

Maybe we should be thinking of all of this in terms that map well to Servo's current architecture.

EDIT: What's really exciting about Servo is that hopefully the architecture's parallelism is really sound, even if certain properties of the current web restrain it. Then we can work on ways to remove those properties, and in Servo it's a simple switch to turn on more and more parallelism.

> In Servo, how do you do interaction with the DOM like `clientHeight`? It just pauses the entire JS engine while it goes and talks to the UI thread?

Yes, but we'd like to experiment with new APIs in the future like `getBoundingClientRectAsync()` that will allow pages to do that kind of thing asynchronously.

I'm also interested in Servo as well. With React Native, we had the liberty to restrict certain paradigms that don't map well to building a parallelizable pipeline of operations. So the JS thread can't synchronously query for layout, which happens in the next stage, and the layout stage can't synchronously block on reading main thread (UIKit etc) values.

By restricting what the programmer can do, it naturally formed a pipeline where layout can be performed in the second stage while JS is determining the next UI update and so on. I'm not sure the extent to which this benefits us right now, but even mobile devices are beginning to have four cores so it seems like a nice property to maintain as long as it doesn't cause problems.

Servo sounds cool, and I'm curious how it takes advantage of a similar architecture but while the programmer has free reign to "clog the pipeline" by synchronous queries on layout etc. I'm sure they've thought of it, I'd just like to hear what they came up with.

I just asked pcwalton in #servo on mozilla and he said they want to introduce async versions of all of the DOM APIs like getBoundingClientRectAsync()

Awesome. That's exactly what we have on React Native :D We want to make sure that when possible, we choose APIs compatible with the future of the web.

We did a bunch of experiments with running React in a web worker. Maybe we should revive them. Because it's React, it ends up being easy and is basically the same setup (serialized events etc) as what React Native does.

Why wasn't this explored more before going full out with React Native? I feel like this would be smart to compare the two: web + webworkers vs native.

There was a lot of hype about hybrid apps "sucking", but I bet it doesn't once you get off the thread.

Trust me, we tried. But as long as image decoding blocks the main thread (Webkit), you are helpless to update the UI thread at a fast pace. We even tried decoding images in a web worker and sending the result back to the main thread - but decoding in JS is very slow so general application throughput suffers (not through fault of JS itself, but because JS lacks SIMD which image decoding algorithms use in order to decode images quickly).

But running React in a web worker is still a really great idea. There would just be a lot of little edge cases to handle (like when you click a link - you need to prevent default of navigation, then send that link click to the web worker to see if your app wants to prevent default, if not, force the redirect back on the main thread). Having tight control over text editing might be challenging as well. These are difficulties, but I suspect they are a fixed set of challenges that are worth the increased parallelism in an increasingly multicore world.

But are web workers currently The Solution To All Our Mobile App Needs? No, but they're a great tool that are highly underutilized by JS developers, and that React is uniquely positioned to take advantage of.

As for events, have the worker send which events it is interested in rather than the reverse as you suggest.

Event handlers are synchronous, and web workers are asynchronous. This means you have to decide whether you will prevent native behaviors before you have a chance to communicate with the web worker.

It's easy to be too aggressive. For example registering a `touchstart` on a DOM node would kill the native behavior and prevent scrolling, even if that's what you wanted. Of course, you could implement scrolling in JavaScript. That's a lot of work and you have to make sure the web worker doesn't block for more than 16ms so that the effect is smooth.

In my experience it's best to use native browser behaviors where possible. You get all sorts of benefits, like scrolling being executed on a separate thread, and don't have to rely on reimplementing everything from the ground up.

Do you have any public examples? I have a use-case where I think this might make a lot of sense, so I've been toying with the idea.

Pedestal Client from Cognitect did this - see https://github.com/pedestal/app-tutorial/wiki/Parallel-Proce... and my notes at http://pchristensen.com/blog/articles/pedestal-tutorial-part... (under "Parallel Processing").

Too bad they stopped working on it.

I've been thinking about this as well yesterday, but to be honest I'm not sure where the benefits are. In theory the browser is doing the rendering in a separate thread anyway, i.e. CSS layout and things like scrolling are handled by the browser. If you're doing a long algorithmic (CPU-intensive) computation in JS then you should probably do that in a WebWorker anyway. Otherwise I'm not sure the overhead is worth it, for example the dragging / touch events should be invoking such a tiny amount of javascript that you shouldn't see much benefit in moving that logic to a separate thread.

Would be interested to see real examples of where this is makes sense.

Disclosure: we're developing an email client using Cordova and purely web tech using our own framework: https://github.com/techlayer/espresso.js

Exactly. I think moving everything but React into the web worker is enough. But that itself isn't easy as far as my limited knowledge, you need to have separate script files. Perhaps a webpack plugin that manages that for you?

I think simply moving any function that takes more than 10ms to a web worker is enough frankly.

Is playcanvas using emscriptem or is it written directly in js? nice project by the way. Also what does the worker do? does it have access to the canvas context or it just does some calculations and communicate with the main thread via serialized data.

Playcanvas is written directly in JS, but does use an emscripten-compiled physics engine (bullet/ammo).

Serialized data. Workers currently can't have access to canvas contexts.

> I'm surprised this hasn't been done, or has it and I just haven't heard about it?

N2O framework might do that.

https://github.com/5HT/n2o

Book:

https://synrc.com/apps/n2o/doc/book.pdf

One pattern you can do is send events to the server, server renders data asynchronously (maintains state for each client in a lightweight process), then ships result back to client via websocket binary frames.

You also decide how much rendering happens where. Can even do whole HTML elements on the server if you want.

Regarding moving script code off the main thread: I wrote a cooperative FRP engine that multiplexes signal graph update recalculation onto Web Workers. UI updates like virtual DOM would live on the main thread.

https://github.com/Johnpmayer/liquid-thorium

IIRC, Web Workers have a really limited subset of functionality available to them, which adds a whole load of gotchas to anything like this.

It is somewhat limited, but using message passing to the main thread, you can render a UI or even use WebGL or Web Audio or anything else. See the link for an example with WebGL.

Oh, I know. But it's rare that a webapp does things computational enough that it's worth the extra overhead - most lagginess is actually at the UI level, in my experience (and why virtual DOMs are so popular now).

I've seen Web Workers used in things like graphing libraries, though.

Someday you'd think they might just go ahead and add multi-threading, shared memory to Javascript? It is definitely a can of worms, but the current web worker situation is just not workable for many use cases.

WPF has a UI and a separate closed off rendering thread, which works quite well performance wise.