News for hackers. Stuff that matters.
Re:Tokio Console by hcarvalhoalves on Friday December 17, 2021 @12:51PM (#29595629)
Very interesting! I can't help but wonder why these technologies (the framework, and accompanying observability) aren't an OS-level feature; wondering if at this point we need an OS at all, and why not ship a minimal kernel + the binary of an application built w/ this to run directly on the VM.
Re:Tokio Console by ReactiveJelly on Friday December 17, 2021 @04:03PM (#29598317)
They're not OS-level because there isn't wide consensus on the right way to do things yet.

Rust, C#, and JS all have similar concepts of async, but they're all slightly different. None of them would be trivial to adapt to other system langs like C and C++ - Rust requires compiler support to take apart async functions and put them back together as state machine, and the others lean on their GC. (And also compiler support, IIRC) I think there is a proposal to add coroutines in the new C++ standard, but I'm not sure how it would be done in the kernel.

And sometimes I see people saying, "async is very bad, just use coroutines." Having only used Lua coroutines, I don't understand what the big difference is supposed to be.

But mostly, these runtimes don't need OS-level support. Async is sort of a way to do concurrency without a kernel-level context switch for every task switch, right? If it's working so well in-process, why involve the OS at all?

> wondering if at this point we need an OS at all

Depends what you mean by OS. If you deploy in a container, of course your OS shares its kernel with the host. But for some user stories, (glares at Android) "OS" means all the software, including a Blink-based web browser, a plethora of GUI programs, and other things that I would rather call a "desktop environment" than an OS.

Re:Tokio Console by paavohtl on Saturday December 18, 2021 @04:02AM (#29603008)
> Rust requires compiler support to take apart async functions and put them back together as state machine, and the others lean on their GC

C# actually does the same thing, though it indeed still relies on GC.

Re:Tokio Console by xtagon on Friday December 17, 2021 @01:13PM (#29595922)
It's been done, check out https://www.nerves-project.org/ for an example of running a minimal kernel + Elixir (BEAM VM) on devices. As I understand it, it's powered by Buildroot and should be possible to do for other VMs/languages. Don't think I've seen anything like this for desktop OS's though!
Re:Tokio Console by pcwalton on Friday December 17, 2021 @04:45PM (#29598725)
In Linux there's work being done on a language-agnostic lightweight threading model, User-Mode Concurrency Groups or UMCG: https://lwn.net/Articles/863386/

One could imagine a similar, runtime-independent console for UMCG. Note, however, that the programming model for such a runtime would be much more similar to 1:1 threading (i.e. blocking I/O with threads) than async/await.

Re:Tokio Console by kristjansson on Friday December 17, 2021 @01:29PM (#29596187)
Unikernels are a thing e.g. https://mirage.io. AFIACT uptake has been ... slow. You give up a lot to erase most or all of your OS, and only the most performance sensitive applications realize a benefit
Re:Tokio Console by im_down_w_otp on Friday December 17, 2021 @08:59PM (#29600777)
That's basically what we did with https://github.com/auxoncorp/ferros, Bundle Rust programs together as tasks to run atop the formally verified seL4 microkernel.
Re:Tokio Console by hayley-patton on Friday December 17, 2021 @07:19PM (#29600010)
Arguably, they are an OS-level feature, in the form of OS threads and your favourite task manager; but it is more expensive to run a very large number of OS threads, compared to async tasks or green threads.

Per Dan Ingalls [1], an operating system is a collection of things that don't fit into a language. There shouldn't be one.

[1] Design Principles Behind Smalltalk <https://www.cs.virginia.edu/~evans/cs655/readings/smalltalk....>

Re:Tokio Console by c-cube on Sunday December 19, 2021 @12:31PM (#29616019)
What about people who want to use several programs on one computer, possibly even written in different languages? This idea is ridiculous unless you think everyone shares your dream of running everything in a smalltalk image.
Re:Tokio Console by hayley-patton on Tuesday December 21, 2021 @02:43AM (#29634990)
I don't think everyone shares _my_ dreams, which mostly don't involve Smalltalk images, but this seems quite feasible. And, from experience, making languages interoperate is a hassle, so "written in different languages" isn't all great.

In the particular case of Smalltalk, the operational(?) semantics of which are specified by a virtual machine, one can just compile targeting the virtual machine. c.f compiling Smalltalk and Java to the Self virtual machine, or all the compilers targeting the JVM nowadays.

Re:Tokio Console by billfruit on Friday December 17, 2021 @01:07PM (#29595848)
Yes truly, is there something like this for linux threads and processes?
Re:Tokio Console by pkaye on Friday December 17, 2021 @01:36PM (#29596296)
The Linux Perf tool is pretty powerful. I've barely just scratched the surface myself.

https://perf.wiki.kernel.org/index.php/Main_Page

As as a side note, Go and pretty good profiling tools. You can see a trace of goroutine execution and why a goroutine got scheduled out and when GC kicks in.

https://about.sourcegraph.com/go/an-introduction-to-go-tool-...

Re:Tokio Console by mycoliza on Saturday December 18, 2021 @05:11PM (#29609058)
A lot of the Tokio console UI was inspired by `htop`, which provides a pretty similar overview of processes and threads. It doesn't really have the same ability to inspect things like `pthread_mutex` and timerfds etc in the same way that the Tokio console can inspect the state of `tokio::sync::Mutex` and `tokio::time::Sleep`, though; although I wonder if something like that could be possible with eBPF...
Re:Tokio Console by erichocean on Friday December 17, 2021 @10:57PM (#29601489)
> wondering if at this point we need an OS at all

People have written unikernels, but they're not that interesting anymore when you can trivially constrain Linux to a single core, keep the remaining cores for you app (no context switching), and keep all of the Linux-y goodness for admin and debugging (SSH, gdb, etc.). All of the performance, none of the admin and deployment headaches.

Basically, unless you truly can't afford that extra core, unikernels are all downside at this point.

Re:Tokio Console by pjmlp on Friday December 17, 2021 @02:11PM (#29596909)
With languages with rich runtimes that is exactly the point.

Actually in a way, POSIX is the missing C's runtime that wasn't made part of ISO C.