I'm curious, would you mind elaborating on your use-cases for sandboxed backend application code? This is clearly the direction in which it's all headed, yet is still mentioned relatively rarely in WASM-related discussions on here.
In general I think being able to run user-supplied code in a sandbox is the most interesting use-case for WASM and backends/Go.
Compiling entire applications to WASM and running them in a WASM runtime is of no interest to me. So for instance WASI isn't actually useful to me since I'd only want to project a very narrow set of APIs into the runtime for the (WASM) code to call. I certainly do not want to give it access to my system and then have to clamp down what access I give it.
I think what the original post points to is an interesting step in the right direction. If someone were to figure out an ergonomic and safe way to support structs, []byte slices as both parameters and return values that would probably open up a lot of new possibilities by lowering the barriers for adoption. If it is Go specific, I can live with that. If it can be done in a manner that supports lots of other languages, that would be fantastic, but to me, right now, it isn't critical.
If it were up to me I'd say forget about WASI for now and solve that problem instead. Yes WASI may be slightly useful some day (I can't see that I'll ever use it), but actually being able to have sandboxed runtimes where I can project in my APIs is vastly more useful.
(I'm not saying WASI is not useful. I'm saying that it is not useful to me. And I think that if projecting Go APIs into a sandbox is easy, then WASI probably isn't going to be important to most people doing backends as they can project more application specific, narrower and more strictly guarded APIs into the sandbox)
Given what you've just described, do you have an opinion on the component model & WIT[0]?
[0]: https://component-model.bytecodealliance.org/design/wit.html...
Using containers to run untrusted third party code is horribly risky. Containers provide nowhere nearly enough isolation and separation.
Anyway, points taken, thanks.
When working with backend systems you generally have 2 choices, a less secure containerized environment that does a very good job of distributing resources or a bulky VM which has a fixed memory cost that you might not need.
WASM sits right in the middle of the two. It has a durable sandbox that prevents bugs in the application logic from exposing the whole machine to exploit (and it itself can be further isolated using the same containerization security practices) but also has the benefits of single app like resource requirements.
Another prime benefit of WASM (not related to the sandboxing) is that WASM is a write once run anywhere standard. That means you don't need a different x86 and ARM wasm deployable. Similar to Java's JAR.
Then don't worry [...] high performance programming" is ill defined. For the term to have any real meaning one has to be more specific about what one tries to achieve.
I don't know what the GC story is like for Go in WASM but it would make me very wary.
The two main use-cases for me would be data transformation and message routing or decision logic that is maintained by the user. That is, the user develops the code to parse custom data formats and emit them in various other data formats, tests and compiles it locally and then uploads it into the backend to be instantiated and run. Ditto for routing/decision logic. (So typical "reactor" usage).
Having third parties produce Go binaries to do this would be resource intensive and awkward.
One of our projects is an IoT platform where we get lots and lots of messages from devices. Some of these devices are very constrained and use, or want to use, space efficient custom data formats. However, they'd like to make the data available in more practical formats for consumers. Hence the need to be able to transform them to one or more other formats in real-time. I'd like to let the customers to be able to write these codecs.
There are a few other uses as well that we are looking at, but common for all of them is that none of them need a generic system interface. Typically you want to project some API(s) into the runtime that the code can call.
(I'm also on the lookout for ways to embed compilers in the backend so customers can edit code in a web interface and compile it in the backend, but embedding compilers has proven to be ... not incredibly fun)
How do you enforce a contract on these WASI artefacts? How do you stop the user doing bad things (like infinite loops or crashing the surrounding program)?
The first issue you bring up is really one about resource limiting. You would want a mechanism to set resource limits (CPU, memory, execution time etc) and then be able to pause/unpause, throttle, terminate, restart or perform some other action when you hit a resource limit. It is safe to assume people will make mistakes. :-)
It has been a while since I looked at runtimes, but I don't think resource limits would necessarily be that hard to implement.
It seems to me that preventing crashing the surrounding program isn't something you can necessarily address in a runtime. Any library or component you use in your Go code can potentially crash the program. And if you project APIs into the sandbox that can be called in ways that crash the surrounding program, that's mostly on you, the programmer, and not really something I think the runtime could reasonably prevent.
Guest crashes become panics the host can recover from.
Finally, you can use a Context to abort execution of the guest, although this option introduces a performance penalty: every backjump must read a counter to periodically poll this context.
That said there are issues when running adversarial code.
The default WASI filesystem implementation is not really meant for sandboxing. When you mount a host folder in the guest, it's possible for the guest to escape the root. You can improve the situation by using virtual files, but that's more work if you want anything sophisticated. WASI is simultaneously too sophisticated to make it hard/slow to emulate+sandbox (symlinks and Unix path resolution), and lacking important features.
There are little to no protections against speculative execution (i.e spectre) attacks. With memory bounds checks implemented in software, you're bound to leak information that way.
So please: don't mix multiple customers custom code in one Go host process. It's not really meant for that.
Allowing your single customer (per host/VM) to customize your product with some plugins, giving some tought about filesystem access (to prevent privilege escalation), is more realistic.
You can always find use-cases where GC'ed languages might be challenging to use. With emphasis on "might", because you kind of have to try it out before you can determine if it is a problem.
There are a lot of high performance applications where they're just fine. For instance if you care more about throughput than latency variance. This is why "high performance" isn't a useful term. You have to be specific about how you measure "performance".
Most of the code for most developers isn't terribly performance critical. Correctness is usually far more important. Most performance tends to be lost due to lack of design talent rather than the language.
For me it is for three reasons: 1) I mostly write things in Go, 2) My main use-cases for WASM are backend applications, 3) being able to run sandboxed code (in backends) would be really, really useful.
Sadly, right now it is rather fiddly to make use of WASM in Go.
> If I want to do high performance programming I don't want to > worry about garbage collection
Then don't worry.
For most applications performance isn't going to be lost due to GC, but due to poor design choices and lack of effort. My observation is that most programmers tend to overestimate their own ability to routinely produce high performance code and under-estimate the cost of producing high performance code. The latter is more important. Writing high performance code (regardless of language/runtime) is time-consuming and tends to require a lot of skill to achieve consistently.
That being said, "high performance programming" is ill defined. For the term to have any real meaning one has to be more specific about what one tries to achieve.
> Go has never focused on binary size and probably never will unless > it actually becomes an issue for Google.
Probably correct. The problems of those who put in the work developing Go is probably going to receive priority. Binary size isn't a huge problem for the things people tend to use Go for.
As an aside; I've been writing software for an embedded Linux platform lately where binary size can become a challenge. I observed that the binaries produced by roughly equivalent C++ programs are about the same size as the Go binaries. Bigger if you take into account that the C++ programs were dynamically linked and the Go programs were statically linked. So your mileage may vary.
It may be that it is possible to introduce tooling and libraries that would allow you to generate WASM output that has a significantly smaller footprint.
> That's the disadvantage of having a corporate BDFL I guess.
No, I don't think it is tied to Google's ownership of Go. As with almost all large open source projects it is about developers and priorities. And a lot of open source gets developed on company time. If you want to help the Go project produce smaller WASM binaries then I don't see why Google would discourage you from contributing. If companies and people do not want to contribute it isn't going to get done.