The performance of the Go runtime with a million blocked goroutines is pretty OK, but its performance with even 1000 runnable goroutines is not great at all. You really need to think about which you are going to have.

Is 1k runnable goroutines materially worse than 1k runnable threads? (Honest question, I don't know myself).

True, but if you regularly have 1000 runnable goroutines then could you not reconfigure your app to have, say, 64 runnable goroutines, and get better throughput? Large numbers of goroutines do seem to be a good fit for problems which are mostly waiting on the network.

The architecture of Go forces you to have 1 goroutine servicing every socket, so the number of runnable goroutines will then be at the mercy of your packet inter-arrival process.

Go does not forces you to do any of that: https://github.com/panjf2000/gnet ( and as you can see it's as fast as the fastest C++ lib )

Go provides go routines as a building block, it's up to you to use that or use something else ( reactor pattern, epoll ect ... )

Aren’t sockets the textbook example of what’s usually blocked? When do you have 1000 sockets all ready to read?

Systems often solicit these kinds of packet storms, for example in root-and-leaf search architectures. See "TCP incast" for writings about this problem.

https://www.usenix.org/system/files/login/articles/chen12-06...

I'm not too familiar with go, but could you make the socket-routines very lightweight? Like dump the work on a threadpool, and then go back to sleep?

Does it? You could implement a classic select-based reader and writer in a single goroutine, to handle many sockets at once.

Please describe how you would actually do that in Go.

Using epoll directly (via syscalls) like here https://github.com/tidwall/evio

My information is outdated, but last I knew, Go always parked a goroutine when making a syscall. Seems like a lot of overhead to read bytes that you know are there from an epolled fd.

You could put epoll fd in non-blocking mode, wrap it with os.NewFile, use SyscallConn() to get syscall.RawConn object, and then use its Read method. Its Read method is special: you can return "not ready", and it will use the Go runtime poller to wait until it's readable, in this particular case effectively putting epoll in a epoll.

In epoll case, using RawConn.Read would look like this:

    err := rawConn.Read(func(fd uintptr) bool {
        nevents, err = syscall.EpollWait(int(fd), events[:], 0)
        if nevents == 0 {
            return false // try again
        }
        return true
    })

Note that using RawConn.Read here is only necessary because epoll needs epoll_wait(2) instead of typical read(2). For ordinary file descriptors, like pipes, etc., setting them to non-blocking mode, wrapping them with os.NewFile, and using its ordinary Read/Write methods is sufficient.

Not exactly. It depends on the syscall. Syscalls that are known to be slow are treated the way you're describing, but syscalls that are often fast are treated optimistically in a way that allows the work queue to be stolen and spawned onto a new thread only if the syscall doesn't return quickly.

I don't know which category epoll fits into, but it seems like it should be treated optimistically, since epoll is used for non-blocking I/O.

https://utcc.utoronto.ca/~cks/space/blog/programming/GoSched...

read(2) has to fit into the "might block" bucket, though, right? It would take a very smart runtime to figure out you just called epoll recently and established it won't block.

> If thread context switch overhead was as low as usermode context switching, there would be no use for coroutines since you could just use threads instead

Two points: a) native threads switching overhead is extremely low, probably lower than usermode Go thread overhead; b) the utility of usermode threads isn't to lower overhead.

It's a given that usermode threads introduce lots of overhead, if only because you're reimplementing the scheduling machinery that already exists in the kernel and that you're already paying the performance penalty for.

But theoretically, you could implement specially tailored scheduling algorithms specific for your app and gain an advantage there - kernel schedulers are general-purpose and can introduce latencies, starvation and other undesired special effects.

P.S. The Linux kernel also has interfaces for changing the thread scheduler algorithms. This gets you 99% of the way there - native threads + round-robin realtime thread scheduler gets you the performance benefits you want without having to rewrite schedulers and context switchers in usermode for every language runtime.

"native threads switching overhead is extremely low, probably lower than usermode Go thread overhead"

This is most likely very wrong.

When you think about in depth, the kernel can context switch between threads essentially for free when returning from a syscall (the work to restore a userland context is the same regardless of which context it is switching to, disregarding cache effects).

In most apps, context switches would happen only due to kernel->user transitions anyway (such as an I/O syscall finishing or a timer triggering), so in most apps kernel context switches are indeed almost free.

To be significantly more efficient than the kernel there should be much more context switches than syscalls, which for non-synthetic programs is mostly only possible under heavy loads that can batch, with suitable I/O interfaces such as io_uring.

It isn't. Kernel context switching is the same thing as language runtime context switching, except you get access to kernel mode and you don't have to keep track of garbage collection stuff.

I thought the major cost was cold cache after switch in both cases?

Kernel context switches are pretty light compared to the following slowdowns due to cache.

Some measurements of context switching: https://eli.thegreenplace.net/2018/measuring-context-switchi...