News for hackers. Stuff that matters.
Re:Choosing between programming in the kernel or in user space by tptacek on Sunday January 21, 2018 @06:04PM (#16200619)
This answer is malpractice on at least two levels:

1. It entertains a performance argument without interrogating the task and environment the appliance is engaged in. You can always in theory make a typical program faster by rewriting it in carefully scheduled assembly. We don't often do that because the performance win would be irrelevant to the problems we're trying to solve.

2. It pretends that the only reliability benefit from moving to u from k is in virtual memory. But, no: the major benefit is from moving from C to something more expressive and safer. The notion of "defect density" captures this argument: with it held constant, a solution that can be expressed in fewer lines of code will have fewer faults. But, of course, defect density between C and a high-level language won't be constant, and it won't favor C, where you can't even count safely with the basic types provided by the language.

Re:Choosing between programming in the kernel or in user space by codemac on Monday January 22, 2018 @12:53AM (#16202210)
Once the driver doesn't exist in the kernel (vfio/iommu), and you have processes and threads and efence/mprotect, this all doesn't make sense for practical purposes. The author's reasoning just doesn't hold up in modern networking and storage appliances when it comes to state of the art performance in Linux userspace.
Re:Choosing between programming in the kernel or in user space by aidenn0 on Monday January 22, 2018 @04:13AM (#16203015)
RE: 2

There is no reason HLLs can't be used in kernel space, is there? Excepting shell-like languages (which require fork, which you typically can't do in a kernel), most embedded kernels provide everything a HLL might expect in kernelspace.

Re:Choosing between programming in the kernel or in user space by koverstreet on Monday January 22, 2018 @12:39AM (#16202170)
Malpractice? Strong words, considering you're mischaracterizing the article.

"Virtual memory—which gives us the user/kernel space split and the process model of programming whereby programs are protected from each other—is just the most pervasive form of isolation."

That's not claiming virtual memory is the only reliability benefit, and you can't expect a short article to be a treatise on all the differences between kernel space and userspace.

Furthermore, the article isn't claiming that kernel programming is _better_ than userspace programming, it's just pointing out that there's a tension between performance and safety - which you acknowledge!

Re:Choosing between programming in the kernel or in user space by tptacek on Monday January 22, 2018 @12:49AM (#16202194)
There isn't necessarily even that tension. Modern high-speed networking does TCP/IP entirely in userspace.
Re:Choosing between programming in the kernel or in user space by koverstreet on Monday January 22, 2018 @12:52AM (#16202203)
When networking is faster in userspace it's because it's a clean slate implementation, not because userspace is faster.
Re:Choosing between programming in the kernel or in user space by JoachimSchipper on Monday January 22, 2018 @02:20AM (#16202501)
A TCP/IP stack isn't fundamentally faster for running in user- or kernelspace, but avoiding user/kernel transitions helps. So maximum-performance implementations that don't need to care about playing nice with others might be best off choosing one or the other and staying there; running in userland is then more convenient (but e.g. the TUX in-kernel webserver exists/existed.)

Mind you, most high-performance software doesn't need to go this far, and stuff like PF_RING / Linux' packet_mmap or "batched" systems calls like recvmmsg() (or even read()) further reduces the number userland/kernel transitions.

Re:Choosing between programming in the kernel or in user space by koverstreet on Monday January 22, 2018 @03:13AM (#16202740)
The vast majority of the time though you need to let multiple processes use the same TCP/IP stack. As soon as you have to do that, moving the TCP/IP stack to userspace means you have to do context switches instead of syscalls.

That is not a good thing if you care about performance.

Re:Choosing between programming in the kernel or in user space by justincormack on Monday January 22, 2018 @04:05AM (#16202977)
The kernel implementation does context switches and syscalls. It depends on your usersoace architecture if you have to do context switches, you get to decide.
Re:Choosing between programming in the kernel or in user space by koverstreet on Monday January 22, 2018 @04:59AM (#16203208)
No. If I use netcat on my Linux machine to connect to a remote machine and send and receive packets, no context switches are involved.

If I want to move the TCP/IP stack to userspace on my Linux machine, how do all the programs I use (firefox, ssh, IRC client) send and receive packets without context switches?

If your answer is to change the userspace architecture and shove everything into one process, you've made isolation _worse_. If you don't, how do you avoid the context switches to talk to the network stack?

The key point is that the network stack has to own shared state, and sending and receiving packets _must_ touch that shared state. If it's accessible to every process (shared memory), you've tossed out isolation and security. If it's not, you have to context switch.

Re:Choosing between programming in the kernel or in user space by lmm on Monday January 22, 2018 @06:37AM (#16203615)
> If your answer is to change the userspace architecture and shove everything into one process, you've made isolation _worse_.

Processes are one way of doing isolation but they're a very crude one (give each thing that needs to be isolated its own address space, swap the whole stack every time you switch tasks). A userspace scheduler (which, sure, would be running within a single process at the UNIX level, but that's the wrong level to be thinking about it) that knows about the details of what it's running can achieve the same level of isolation in a more lightweight way.

Re:Choosing between programming in the kernel or in user space by JoachimSchipper on Monday January 22, 2018 @02:35PM (#16207193)
netcat does switch into and out of the kernel; you're otherwise correct that userland networking pulls towards a one-process appliance (hence my original "... that don't need to care about playing nice with others").
Re:Choosing between programming in the kernel or in user space by pjmlp on Monday January 22, 2018 @02:49AM (#16202623)
Microkernel operating systems, like L4 and QNX beg to differ.
Re:Choosing between programming in the kernel or in user space by koverstreet on Monday January 22, 2018 @03:32AM (#16202831)
What is your claim?

Are you going to claim they do networking faster than OpenBSD?

Re:Choosing between programming in the kernel or in user space by pjmlp on Monday January 22, 2018 @03:36AM (#16202851)
Given the actual use in production systems, I guess so.

How fast does OpenBSD do radio networking on mobile phones?

How fast does OpenBSD do CAN networking on car systems and factory installations, with real time constraints?

Re:Choosing between programming in the kernel or in user space by koverstreet on Monday January 22, 2018 @03:56AM (#16202937)
You're arguing a strawman.
Re:Choosing between programming in the kernel or in user space by pjmlp on Monday January 22, 2018 @04:09AM (#16202994)
No, if OpenBSD was actually better for those use cases I guess companies would have noticed it already, instead of paying for microkernels with slower performance.
Re:Choosing between programming in the kernel or in user space by koverstreet on Monday January 22, 2018 @04:46AM (#16203162)
They absolutely do have worse networking performance. Just not enough to matter for those applications.
Re:Choosing between programming in the kernel or in user space by pjmlp on Monday January 22, 2018 @04:51AM (#16203179)
If OpenBSD is so great serving network requests why isn't it taking over the world of cloud services?
Re:Choosing between programming in the kernel or in user space by lmm on Monday January 22, 2018 @06:39AM (#16203626)
Because cloud providers would rather pay for a little more hardware than have to hire or train an ops team that understands OpenBSD?