Re:Wasmer: A Python library for executing WebAssembly binaries
by galacticdessert on Tuesday April 16, 2019 @02:08AM
(#19671485)
Looks good! Just for my understanding, what would be the advantage of working with wasm binaries instead of, let's say, C external libraries or DLLs via ctypes?
I can think of:
1. Portability of the wasm binaries
Is easier interoperability between data types an advantage as well? Ctypes can be a nightmare when passing around structs and pointers (and pointers of pointers, etc.)
I'm hooking python scripts as custom-build-jobs into C/C++ projects which (for instance) takes care of compiling GLSL shader code into HLSL (D3D), MSL (Metal) and various GLSL versions (for GLES2, GLES3 and desktop GL).
Khronos provides C++ projects which help with this stuff:
So I'm using Python to hook my custom build jobs into the C/C++ build system (via cmake), but I still need to invoke native command line tools (or DLLs) which wrap around those Khronos libraries. Currently this means building (at least) 3 executables for Windows, macOS and Linux, and "distribute" those precompiled exes with my python code.
It looks like with this python+wasmer solution, I can instead compile the Khronos libraries into WASM modules, and the result runs "anywhere" where wasmer runs (e.g. also Raspberry Pi, BSDs etc...).
Re:Wasmer: A Python library for executing WebAssembly binaries
by JackC on Tuesday April 16, 2019 @09:48AM
(#19673540)
Portability might also bring along with it easier reproducible builds? Right now I believe if you use wheels, you basically have to trust that the distributed binary matches the source code. I saw a case where a package was taken over by a new maintainer who released wheels for an existing source release, and everyone basically shrugged and said, eh, this is probably fine, because there was no easy way to check. It sounds much easier to build an automated system that could check if wasmer-based packages are faithfully compiled from source, because there's only one compile target.
Sandboxing also sounds valuable, as mentioned by a sibling. I'm thinking of projects like imagemagick that run all kinds of untrusted input through all kinds of processing. It would be nice if there was a simple default way to write data processing libraries the way djb would write them.[1]
Re:Wasmer: A Python library for executing WebAssembly binaries
by make3 on Tuesday April 16, 2019 @02:36AM
(#19671587)
My first impression is that the goal of this project is to run binaries that were likely built for other purposes like the web, not that they actually think that making libraries into webasm and running them into python has any particular advantages vs other options that already exist
Re:Wasmer: A Python library for executing WebAssembly binaries
by indentit on Tuesday April 16, 2019 @01:10AM
(#19671283)
This looks great, I like the simple examples which show how to get a string out from the web assembly module.
One thing I didn't see in the readme or examples, however, is a way to import python functions to the web assembly modules. Is it possible, and just missing from the documentation atm?
Re:Wasmer: A Python library for executing WebAssembly binaries
by Hywan on Tuesday April 16, 2019 @01:52AM
(#19671420)
We don't support that yet, but we are working on it. It's not that easy even if the theory is kind of simple to explain. The implementation in the different backends is mostly the challenge. The Python extension uses the Wasmer runtime (https://github.com/wasmerio/wasmer) which supports 3 backends to generate executable code: Singlepass, Cranelift, and LLVM.
Cranelift is the easiest one to embed, and it brings a great balance between compilation time and execution time. That's the one used in the Python extension right now.
The “import functions” feature is also missing in the PHP extension (https://github.com/wasmerio/php-ext-wasm), but yeah, as I said, we are working on it. It's coming soon!
Re:Wasmer: A Python library for executing WebAssembly binaries
by Hywan on Tuesday April 16, 2019 @02:32PM
(#19676090)
That's true. However, it allows to integrate your Wasm program to your favorite framework more easily. It's likely to be a small surface of code to review carefully. It's always a balance :-).
Re:Wasmer: A Python library for executing WebAssembly binaries
by tracker1 on Tuesday April 16, 2019 @03:42PM
(#19676639)
It's often for code that is asynchronous that will not return an answer immediately... the function passing is most often used to send a callback for these situations.
Re:Wasmer: A Python library for executing WebAssembly binaries
by jononor on Thursday April 18, 2019 @10:14AM
(#19691136)
A state-machine, generator or coroutine can be used instead of callbacks.
I mostly prefer only having one direction of who-calls-who when combining multiple languages. Though I understand that for a general purpose system like Wasmer it makes sense to support everything, so that users can choose the architecture.
Re:Wasmer: A Python library for executing WebAssembly binaries
by indentit on Tuesday April 16, 2019 @02:15AM
(#19671507)
understood, thanks for the explanation - keep up the great work! is there anything external contributors can do to help? :)
I didn't find any issues labelled "good first issue" but I did find a related issue[0]
Re:Wasmer: A Python library for executing WebAssembly binaries
by Hywan on Tuesday April 16, 2019 @02:34PM
(#19676100)
The best way to help right now is to test it, and report any issues! I've been able to try it on Windows so far for instance.
Also the API isn't complete. We need to be able to grow the memory for instance. We need to understand the community needs to define a roadmap for the community itself.
Re:Wasmer: A Python library for executing WebAssembly binaries
by KloudTrader on Tuesday April 16, 2019 @12:10AM
(#19671078)
This looks really fun to use for algorithmic trading (we have to deal with and sandbox client code not written by us here at KloudTrader). Wonder how hard it is to integrate with numpy etc.
Re:Wasmer: A Python library for executing WebAssembly binaries
by btown on Tuesday April 16, 2019 @01:06AM
(#19671270)
I'm sure there are better ways to do this, but if you really want Python client code -> WASM -> Python host, I'd imagine you could probably run https://github.com/iodide-project/pyodide within Wasmer. But you'd likely lose a lot of the performance benefits of numpy etc. being compiled for the exact intrinsics supported by your host CPU. Container-level sandboxing still sounds like a superior bet.
Re:Wasmer: A Python library for executing WebAssembly binaries
by snek on Tuesday April 16, 2019 @12:35PM
(#19675086)
It's a side effect of the design of WebAssembly, as long as wasmer itself has a conformant implementation (the WASM spec itself is already formally verified).
I can think of: 1. Portability of the wasm binaries
Is easier interoperability between data types an advantage as well? Ctypes can be a nightmare when passing around structs and pointers (and pointers of pointers, etc.)
Shader-cross-compilation for 3D frameworks:
https://floooh.github.io/2017/05/15/oryol-spirv.html
I'm hooking python scripts as custom-build-jobs into C/C++ projects which (for instance) takes care of compiling GLSL shader code into HLSL (D3D), MSL (Metal) and various GLSL versions (for GLES2, GLES3 and desktop GL).
Khronos provides C++ projects which help with this stuff:
https://github.com/KhronosGroup/glslang
https://github.com/KhronosGroup/SPIRV-Cross
So I'm using Python to hook my custom build jobs into the C/C++ build system (via cmake), but I still need to invoke native command line tools (or DLLs) which wrap around those Khronos libraries. Currently this means building (at least) 3 executables for Windows, macOS and Linux, and "distribute" those precompiled exes with my python code.
It looks like with this python+wasmer solution, I can instead compile the Khronos libraries into WASM modules, and the result runs "anywhere" where wasmer runs (e.g. also Raspberry Pi, BSDs etc...).
That's really cool stuff :)
Sandboxing also sounds valuable, as mentioned by a sibling. I'm thinking of projects like imagemagick that run all kinds of untrusted input through all kinds of processing. It would be nice if there was a simple default way to write data processing libraries the way djb would write them.[1]
[1] https://cr.yp.to/qmail/qmailsec-20071101.pdf
Cranelift is the easiest one to embed, and it brings a great balance between compilation time and execution time. That's the one used in the Python extension right now.
The “import functions” feature is also missing in the PHP extension (https://github.com/wasmerio/php-ext-wasm), but yeah, as I said, we are working on it. It's coming soon!
I mostly prefer only having one direction of who-calls-who when combining multiple languages. Though I understand that for a general purpose system like Wasmer it makes sense to support everything, so that users can choose the architecture.
[0]: https://github.com/wasmerio/wasmer/issues/267
Also the API isn't complete. We need to be able to grow the memory for instance. We need to understand the community needs to define a roadmap for the community itself.
https://github.com/wasmerio/python-ext-wasm/blob/master/src/...
but in theory it shouldn't be too hard™
That's a very bold statement, I see no backup of this in the readme.