News for hackers. Stuff that matters.
Re:JavaScript in Parallel: Web Workers and SharedArrayBuffer by impostervt on Friday March 03, 2017 @07:10PM (#13788175)
What do people actually use Web Workers for? The examples I've seen, including this one, seem contrived. I keep hoping they'll change it to allow background image manipulation, but I haven't seen much real progress in that front.
Re:JavaScript in Parallel: Web Workers and SharedArrayBuffer by Klathmon on Friday March 03, 2017 @08:39PM (#13788601)
I've used it for background image manipulation before.

Paint the image to a canvas, then grab the imagedata off of it, split it into as many parts as you have threads, then use the "transferrableObjects" property of postMessage to zero-copy transfer the data to each worker to be processed, transferred back, and re-stitched together.

It's pretty powerful and suprisingly easy to work with once you understand it.

[0] is a snippet from the code, but be gentile... It was a personal project where I was trying out polymer 0.5 and made a lot of questionable design choices...

Also, I've heard of the idea of using webworkers as a "first class" platform. That is do all of the core parts of your application in them and only use the "main" thread as a "ui" thread. I haven't gotten a chance to try it out, but it seems like a great idea that could really work well in some SPAs.

[0] https://github.com/Klathmon/stitchpics/blob/master/app/eleme...

Re:JavaScript in Parallel: Web Workers and SharedArrayBuffer by bartread on Saturday March 04, 2017 @06:42AM (#13790460)
That's interesting but, AFAIK (and, believe me, I'd be happy to be corrected), what you can't do is create a canvas element (even one not attached to the DOM), and paint directly to it using the standard 2D context and drawing primitives.

Like I say, more than happy to be corrected, because that sort of thing would be incredibly helpful. Really, anything that lets you mess with a disconnected DOM in the background and then attach it in the foreground could be useful but (and, again, I'm very happy to be corrected), I don't think you can do this.

Re:JavaScript in Parallel: Web Workers and SharedArrayBuffer by vanderZwan on Saturday March 04, 2017 @07:12AM (#13790564)
You're talking about OffscreenCanvas. It's available in Firefox and nowhere else:

https://developer.mozilla.org/en-US/docs/Web/API/OffscreenCa...

Having said that, if you want fast image manipulation, you're probably better off doing direct manipulation of Uint8Clamped arrays anyway, since that can be much faster.

Re:JavaScript in Parallel: Web Workers and SharedArrayBuffer by bartread on Tuesday March 07, 2017 @09:53AM (#13810699)
Yeah, exactly that, although sadly Firefox represents a minority of traffic for me so OffscreenCanvas isn't a realistic option.

To your point about direct manipulation, I think you may be right. I'm talking about drawing rather than image processing, and no way I want to effectively re-implement a bunch of Canvas2D primitives, but what I could do is draw the image once on the foreground thread, extract the raw bytes, and then rotate those using image processing into a bunch of target arrays in the background. I'd then pass those arrays back to the foreground and create a set of corresponding images.

The only issue here, and I'd need to benchmark this, is that it's not clear to me how much of the time is spent drawing versus actually creating the images. If creating images is expensive this might not yield much of a gain.

The upshot is that this starts to sound like quite a bit of work when, what I could do, and which would be much simpler, is reduce the fidelity slightly: e.g., by rendering only 180 or 120 images, and tweaking the minimum angular velocity appropriately to avoid jerky animation.

And then I suppose I'm back to the point that somebody else made, which is that Web Workers may not be that useful in the real world. :/

Re:JavaScript in Parallel: Web Workers and SharedArrayBuffer by Klathmon on Saturday March 04, 2017 @07:50AM (#13790675)
And if you can get away with it and don't mind a lot of bit shifting, it's even better working with the Uint32array which packs all 3 colors and alpha into one element which reduces your loops by 4X
Re:JavaScript in Parallel: Web Workers and SharedArrayBuffer by vanderZwan on Saturday March 04, 2017 @09:27AM (#13790993)
Because matching the native 32 word size is better for the prefetcher, right?

Wouldn't most CPU's these days be smart enough to detect advancing the index by 4, and then using offsets?

So:

    for(let i = 0; i < someUint8Array.length; i += 4){
      let R = someUint8Array[i], G = someUint8Array[i+1],
        B = someUint8Array[i+2], A = someUint8Array[i+3];
      // ... manipulations here
    }
Re:JavaScript in Parallel: Web Workers and SharedArrayBuffer by Klathmon on Saturday March 04, 2017 @09:50AM (#13791099)
I'm honestly not sure why it is, but across all browsers on both ARM and x86_64 arches it was almost 3x faster than doing what you wrote.

I have a feeling it's more of a JS JIT thing than a CPU prefetcher thing, but honestly I'm not really sure.

In my program I linked above, it was actually faster to use Uint32array everywhere and then use functions to pull the 4 color values from it and another function to push the 4 values back to a uint32.

Granted, it's been over a year since I last benchmarked that code, but I did reuse some of the image code recently and found iterating over a Uint32array to be significantly faster. (And funnily enough, manually unrolling the loop of Uint32array to something similar to what you wrote gave an additional small performance boost, but it was small enough to be not worth the extra weirdness in the code to me)

Re:JavaScript in Parallel: Web Workers and SharedArrayBuffer by vanderZwan on Saturday March 04, 2017 @10:08AM (#13791158)
Thanks for the info, that might save me some benchmarking time myself in the near future ;)
Re:JavaScript in Parallel: Web Workers and SharedArrayBuffer by Klathmon on Saturday March 04, 2017 @10:21AM (#13791213)
If it helps, I took the class I made that converts between 4 Uint8Clamped values to a Uint32 value and vice versa into an NPM package at [0].

At the very least it can show you some of the gotchas with bit shifting in JS (like how values often look negative until they are placed into a Uint32array then they become positive integers, and how you need to check for endianness)

[0] https://github.com/Klathmon/BitPacker.js

Re:JavaScript in Parallel: Web Workers and SharedArrayBuffer by vanderZwan on Saturday March 04, 2017 @10:33AM (#13791268)
Would not have thought of checking for endianness, thanks!
Re:JavaScript in Parallel: Web Workers and SharedArrayBuffer by Klathmon on Saturday March 04, 2017 @07:53AM (#13790690)
As the other commenter said, you are right that you can't work with canvas in the worker, but you can work directly with the array of image data which you'd often need to anyway for much processing.

I saw a library a while ago that was trying to re-implement the canvas primitives using only typed arrays so it was worker safe but I'm not sure what happened to it.

Re:JavaScript in Parallel: Web Workers and SharedArrayBuffer by bartread on Tuesday March 07, 2017 @09:56AM (#13810734)
Thanks - that might be interesting. See my comment above but, yes, I'd quite like to avoid implementing those primitives. I could get round it by drawing one image, and then processing the bytes for the rotations, but:

- This might lead to the odd jaggy artifact because I'm rotating bitmaps rather than vectors,

- It might not speed things up that much because I still need to create images from these arrays,

- It starts to seem like quite a lot of work; maybe I'd be better off reducing the number of images slightly, and compensating by speeding up the minimum angular velocity.

You can see what I'm up against at https://arcade.ly/games/asteroids/. (Also, see my comment above, where I've made substantially the same points.)

Re:JavaScript in Parallel: Web Workers and SharedArrayBuffer by phpnode on Friday March 03, 2017 @07:19PM (#13788218)
Anything that does significant work that would otherwise result in an unresponsive UI. For example code on the following page is compiled as you type, without a webworker this would give a very horrible user experience because compilation can take a long time for very large input - https://codemix.github.io/flow-runtime/#/try
Re:JavaScript in Parallel: Web Workers and SharedArrayBuffer by ridiculous_fish on Friday March 03, 2017 @07:52PM (#13788403)
I used WebWorkers for some math - basically lots of Fourier transforms, which took a few seconds to complete.

I expected to instantiate a WebWorker with a function, and was surprised to discover you have to give it a separate JavaScript file. It is not easy to adopt.

Re:JavaScript in Parallel: Web Workers and SharedArrayBuffer by gp7 on Friday March 03, 2017 @08:10PM (#13788484)
You can just call toString on a function and Blob the string. You have to keep your head straight about the fact that scopes won't transfer (no closures), which is probably why the API doesn't do it for you.
Re:JavaScript in Parallel: Web Workers and SharedArrayBuffer by bartread on Saturday March 04, 2017 @06:47AM (#13790485)
Yeah, this ticked me off as well because the use of Web Workers feels extremely un-idiomatic, but what you've said makes sense. If you just passed a function it might dupe people into thinking closures would work or, and this is perhaps worse, force the standard down the route of copying all the values available in the current closure into the worker's scope, which just sounds like a terrible idea to me.
Re:JavaScript in Parallel: Web Workers and SharedArrayBuffer by ridiculous_fish on Friday March 03, 2017 @08:54PM (#13788654)
This didn't work in my case because I had references to other functions, for complex number handling and the like. I ended up just loading the whole minified .js file again in the Worker, which worked though it felt ridiculous.
Re:JavaScript in Parallel: Web Workers and SharedArrayBuffer by btbuildem on Saturday March 04, 2017 @12:42AM (#13789501)
One of the native functions available in a web worker is importScript - it allows to load include other scripts (eg lodash) into the worker scope.
Re:JavaScript in Parallel: Web Workers and SharedArrayBuffer by vanderZwan on Saturday March 04, 2017 @07:22AM (#13790584)
I wonder if bundlers like WebPack could make life simpler here..
Re:JavaScript in Parallel: Web Workers and SharedArrayBuffer by Klathmon on Saturday March 04, 2017 @07:54AM (#13790697)
It really really does. Webpack's worker stuff make it so stupid simple that i was kicking myself for not using it a long time ago.
Re:JavaScript in Parallel: Web Workers and SharedArrayBuffer by gliese1337 on Friday March 03, 2017 @10:57PM (#13789145)
I've used them to do real-time audio and video transcoding, and for doing (non-security sensitive) hashing and decryption operations.

They are an excellent fit for anything that requires heavy number-crunching, and little or no DOM manipulation or network communication.

Re:JavaScript in Parallel: Web Workers and SharedArrayBuffer by avgp on Saturday March 04, 2017 @10:32AM (#13791261)
We are experimenting with using SharedArrayBuffers and fetch to stream data, process it and fill it into a SharedArrayBuffer to then render with WebGL in the main thread. This has been working pretty great so far, but we'll have to wait for SharedArrayBuffer to get wider adoption.

Another thing was to offload tasks such as encoding GIF frames, which is also working pretty well.

Re:JavaScript in Parallel: Web Workers and SharedArrayBuffer by vanderZwan on Friday March 03, 2017 @07:47PM (#13788376)
Well, I'm not using them yet, but I working on a SPA where I have multiple react-virtualized-select[0] components, each with thousands of options. Creating the fastFilter options[1] for those takes multiple seconds. Although I only do it once and then re-use it, that still freezes up the page at the start, so if I could do that in parallel without freezing the UI that would improve the app for sure.

I guess for off-screen image manipulation (if for whatever reason you can't use WebGL and shaders) you'd have to resort to manual pixel manipulation by sending Uint8ClampedArrays back and forth. At least they're transferable by default.

[0] https://github.com/bvaughn/react-virtualized-select

[1] https://github.com/bvaughn/react-select-fast-filter-options

Re:JavaScript in Parallel: Web Workers and SharedArrayBuffer by joshontheweb on Friday March 03, 2017 @08:01PM (#13788444)
I run a timeout function that checks for delays in execution and I cross reference this with what is running at the time and move that to a web worker. In my cause this ends up being when I do heavy lifting like merging lots of Float32Arrays (audio stream data) or doing audio encoding.

Edit: Shameless Recruiting Plug. If any of you out there are js performance pros and well acquainted with this sort of thing, please reach out as I'm looking for help optimizing further and I'm a bit out of my depth on some of it. Currently this would be a contract position but could grow to something more if that is desired.

Re:JavaScript in Parallel: Web Workers and SharedArrayBuffer by pesfandiar on Friday March 03, 2017 @08:00PM (#13788441)
I used it to create a sandbox for an online JavaScript coding practice site (if you're interested in the write-up: http://www.pesfandiar.com/blog/2016/05/12/javascript-online-...). I should say it's a very niche use case and the sandbox is not exactly secure, but it's nice to have a separate disposable environment to run your scripts.
Re:JavaScript in Parallel: Web Workers and SharedArrayBuffer by throwanem on Friday March 03, 2017 @10:35PM (#13789063)
I used them last year for parsing and analyzing data from 250-300M XML files, to fulfill a business need. Took a couple of days to implement and worked quite nicely.
Re:JavaScript in Parallel: Web Workers and SharedArrayBuffer by driverdan on Saturday March 04, 2017 @11:05AM (#13791432)
A few years back I built an app that took in data from APIs and did a lot of calculations for visualization. The page could take 5+ sec to render for large data sets, freezing everything while loading. If I built it today I'd use Web Workers.
Re:JavaScript in Parallel: Web Workers and SharedArrayBuffer by jhgg on Saturday March 04, 2017 @01:44PM (#13792233)
We are experimenting with webworkers to power a very complicated autocomplete and scoring system in our client. So far so good. We're able to keep the UI running at 60fps while we match, score and sort results in a web-worker.
Re:JavaScript in Parallel: Web Workers and SharedArrayBuffer by nol13 on Saturday March 04, 2017 @08:38AM (#13790836)
Extracting large .zip's of images to process before uploading, reading and writing large excel files.

Basically to move as much processing client side as possible/reasonable so the server doesn't have to do it.

Re:JavaScript in Parallel: Web Workers and SharedArrayBuffer by rcfox on Friday March 03, 2017 @08:17PM (#13788518)
I work on a browser-based electronics design tool. We use web workers when taking large, complex polygons and transforming them into a bunch of triangles.
Re:JavaScript in Parallel: Web Workers and SharedArrayBuffer by SirensOfTitan on Friday March 03, 2017 @07:22PM (#13788243)
Maybe heavy JSON parsing? Better to parse off UI-thread than drop a frame.

* Did not read article yet and am not sure about SharedArray structure.

Re:JavaScript in Parallel: Web Workers and SharedArrayBuffer by ww520 on Friday March 03, 2017 @08:55PM (#13788660)
Build thumbnail of images, encode audio/video, upload a file in background. Anything that takes more than a few seconds.
Re:JavaScript in Parallel: Web Workers and SharedArrayBuffer by colordrops on Friday March 03, 2017 @09:15PM (#13788739)
In an embedded application with low resources you could push processing into a worker if your CPU is multi core.
Re:JavaScript in Parallel: Web Workers and SharedArrayBuffer by bhouston on Friday March 03, 2017 @08:15PM (#13788512)
Clara.io users web workers to decompression mesh data in parallel.