News for hackers. Stuff that matters.
Re:CPython's main branch running in the browser with WebAsse... by ridiculous_fish on Monday November 29, 2021 @12:36AM (#29376774)
One of the painful aspects of WASM is there's no blocking calls. You can't say "wait for the next event;" instead you must return to the outermost event loop, and wait to be called back.

How does Python-in-WASM work around that? For example, how does `for line in sys.stdin:` work if you can't actually block on stdin?

Emscripten has some support for this via the "asyncify" transform, which layers additional control flow to enable return all the way up the call stack, and then "rewind" back down into it. But this bloats the code (and is also buggy) so maybe it's not being used.

Re:CPython's main branch running in the browser with WebAsse... by ethanhsmith on Monday November 29, 2021 @02:18AM (#29377271)
Yes, currently input goes into a propmpt() and it doesn't output anything unless you hit "Cancel" on the prompt, definitely a bad time.

Python allows you to reach in and replace the core interpreter loop, so this may be an avenue to have our own asyncify-like function pop out to JS land and restore state correctly (which we can be smart about since we are the interpreter).

It may also be possible to write something that runs Python in a webworker and communicate with it over a sharedarraybuffer, but that I'm a bit more hazy on. Pyodide has some discussion of this in https://github.com/pyodide/pyodide/issues/1219 and https://github.com/pyodide/pyodide/issues/1503.

This is definitely the hardest part of getting Python to work. Well, hardest after the hardest part of building a compiler toolchain like Emscripten :)

Re:CPython's main branch running in the browser with WebAsse... by benno128 on Monday November 29, 2021 @06:10AM (#29378454)
In WebAssembly.sh (https://github.com/wasmerio/webassembly.sh) they run WASM binaries in a Web Worker and then use `SharedArrayBuffer` to block the WebWorker while the main thread does some work (e.g. collect input). You could use a similar solution.

When building Runno (https://runno.dev) I forked off that project and did a bunch of other things on top to get blocking to work in Safari and non-cross-origin-isolated contexts.

Ultimately I think it's JavaScript's (or whichever host language) responsibility to block when the binary calls out (if that is the expected semantics).

Re:CPython's main branch running in the browser with WebAsse... by est31 on Monday November 29, 2021 @10:16AM (#29380780)
Does the SharedArrayBuffer approach offer any way to tell the OS scheduler to wait? Because the only waiting method I know of is busy waiting, aka wasting cycles at 100% of a CPU core. In normal processes you can call a sleep function, which saves CPU cycles, but there is no synchronous method for that available in javascript.
Re:CPython's main branch running in the browser with WebAsse... by benno128 on Monday November 29, 2021 @02:39PM (#29384224)
It's part of the Atomics feature set (https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...). You have a shared chunk of memory and you say "please sleep until this memory changes". Then it's up to the host to change that memory.
Re:CPython's main branch running in the browser with WebAsse... by est31 on Tuesday November 30, 2021 @02:35AM (#29389839)
Oh that's pretty neat, didn't know about that.
Re:CPython's main branch running in the browser with WebAsse... by bkolobara on Monday November 29, 2021 @02:10AM (#29377220)
It's on the embedder (Wasm VM) to provide this functionality. I'm working on a Wasm runtime [0] that is written in Rust and uses stack switching to allow you to call Rust async functions as if they were blocking. This keeps the Wasm bytecode simple (blocking), but at the same time provides high performance i/o.

There is also a proposal to bring stack switching to the browser.

[0]: https://github.com/lunatic-solutions/lunatic

Re:CPython's main branch running in the browser with WebAsse... by dosshell on Monday November 29, 2021 @02:16AM (#29377256)
What is the difference between "stack switching" and a fiber based approach?
Re:CPython's main branch running in the browser with WebAsse... by bkolobara on Monday November 29, 2021 @05:12AM (#29378117)
Fibers are usually implemented with stack switching. It should be the same approach.
Re:CPython's main branch running in the browser with WebAsse... by samwillis on Monday November 29, 2021 @03:05AM (#29377444)
I know Absurd SQL[0] uses SharedArrayBuffer and Atomics to turn the async IndexDB into sync for use by Wasm. I wander if it’s possible to use that here too although it’s obviously a little different?

0: https://github.com/jlongster/absurd-sql

1: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

2: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

Re:CPython's main branch running in the browser with WebAsse... by bayesian_horse on Monday November 29, 2021 @02:11AM (#29377228)
For Pyodide I made a proof of concept where you can run async/await with promises.

If people actually develop stuff in Python for the web, they should do something like that.

Re:CPython's main branch running in the browser with WebAsse... by trothamel on Monday November 29, 2021 @02:15AM (#29377248)
In my experience, asyncify works pretty well everywhere but Safari on macOS/ARM or ios/ARM, where the stack sizes are too small to be useful. You do want to be a bit careful about where you block, which can minimize the number of functions that need to be transformed.
Re:CPython's main branch running in the browser with WebAsse... by azakai on Monday November 29, 2021 @11:17AM (#29381570)
> and is also buggy

If you've found any bugs in Asyncify please file them! There are no open issues atm about any general bugs, aside from some corner cases with features like dynamic linking.

Re:CPython's main branch running in the browser with WebAsse... by lights0123 on Monday November 29, 2021 @01:08PM (#29383019)
You can use Atomics.wait() to block a non-main thread.