Re:Unikernels: Rise of the Library Hypervisor
by jzelinskie on Saturday October 08, 2016 @02:24PM
(#12667982)
This presentation looks like it's mostly about how Docker Inc leveraged software from their MirageOS acquisiton while writing Docker For Mac. For the unfamiliar, Docker For Mac is an attempt to make the Docker Engine appear native for macOS users by running an Alpine Linux VM on top of macOS's native Hypervisor.framework.
From what I can tell from this talk and from using the software, the only usage of "unikernel infrastructure" is essentially to hack around elevating privileges for networking in macOS. It sounds like MirageOS was only used because it was a reasonably portable network stack.
Re:Unikernels: Rise of the Library Hypervisor
by avsm on Monday October 10, 2016 @07:12AM
(#12676351)
> the only usage of "unikernel infrastructure" is essentially to hack around elevating privileges for networking in macOS
(author of the talk here)
That's correct, although I view that in a less dismissive light than your comment above suggests :-)
The "unikernel infrastructure" movement hinges on turning as many systems layers as we can into libraries, so that they can be repurposed in the future in different contexts. This can also include using them in _exactly_ the same way as you use systems today (e.g. compiling unikernels to Linux binaries pointing at a socket stack).
But when a problem comes along that requires challenging conventional layering, having unikernel libraries is a game changer. Without this, we would have to maintain a much more heavyweight emulation system, which in turn drives up complexity and resource usage. In the D4Mac/Win case, putting in the VPNKit library layer gives us a lot of flexibility in mapping to existing socket stacks, and also lets us open a broader conversation about what network bridging should look like in the next generation OS stack. Whatever that answer is, we know that solution isn't what we are forced to live with today -- root level privilege and importable kernel modules required to shift Ethernet packets around. Perhaps the answer is something more like FreeBSD's declarative Netgraph framework, or an OSX-style launchd socket registration system that works across higher level protocols including TLS, or something else entirely different. But until then, the unikernel infrastructure lets us solve the immediate systems problem at hand without letting that design decision vomit all over the rest of the application stack and burden us with a ton of technical debt.
> It sounds like MirageOS was only used because it was a reasonably portable network stack
It also happened to be the one that many of the team had authored, and written in a high-level enough style that we knew how to invert it. Nothing dramatic here, just picking the shortest solution to get shipping software that solved real user problems (in this case, VPN and firewall support on OSX/Win).
I expect that pulling off the same trick using several other user-level network stacks is also not hugely difficult, but I haven't used lwIP for some years to find out. The Rump Kernel NetBSD stack is probably the best alternative to start with, or the HalVM TCP/IP stack. The IncludeOS C++ one is also rapidly becoming a mature candidate.
Re:Unikernels: Rise of the Library Hypervisor
by chucky_z on Saturday October 08, 2016 @12:03PM
(#12667396)
Is this a presentation on a unikernel that can run docker (MirageOS/Hyperkit)?
Interesting, I was looking into running single daemons (e.g.: nginx) as a unikernel (through rump kernels). This is certainly something different!
Edit: doing some research it simply seems that Hyperkit/MirageOS are supplementing docker (via Alpine Linux). I may still be missing something just by looking at the slides.
Re:Unikernels: Rise of the Library Hypervisor
by avsm on Monday October 10, 2016 @07:17AM
(#12676364)
(author of the talk here)
This talk was more of a roundup of interesting uses of library hypervisors. The two are:
- the HyperKit/MirageOS use in Docker for Mac and how it can augment a native client application
- the far more general new Solo5 backend that IBM/Docker/Cambridge have contributed to MirageOS3, which shows how to use a library hypervisor to build a very Unix-like unikernel experience. MirageOS developers have also taken advantage of the libraries to add support for FreeBSD/bhyve very easily [1], so these new backends and libraries are increasing MirageOS library portability quite a bit.
(I've just come back from Berlin and Docker Summit and have a ton of interesting questions and FAQs from attendees there. Will try to write up a blog post expanding on this presentation ahead of the MirageOS3 beta!)
From what I can tell from this talk and from using the software, the only usage of "unikernel infrastructure" is essentially to hack around elevating privileges for networking in macOS. It sounds like MirageOS was only used because it was a reasonably portable network stack.
(author of the talk here)
That's correct, although I view that in a less dismissive light than your comment above suggests :-)
The "unikernel infrastructure" movement hinges on turning as many systems layers as we can into libraries, so that they can be repurposed in the future in different contexts. This can also include using them in _exactly_ the same way as you use systems today (e.g. compiling unikernels to Linux binaries pointing at a socket stack).
But when a problem comes along that requires challenging conventional layering, having unikernel libraries is a game changer. Without this, we would have to maintain a much more heavyweight emulation system, which in turn drives up complexity and resource usage. In the D4Mac/Win case, putting in the VPNKit library layer gives us a lot of flexibility in mapping to existing socket stacks, and also lets us open a broader conversation about what network bridging should look like in the next generation OS stack. Whatever that answer is, we know that solution isn't what we are forced to live with today -- root level privilege and importable kernel modules required to shift Ethernet packets around. Perhaps the answer is something more like FreeBSD's declarative Netgraph framework, or an OSX-style launchd socket registration system that works across higher level protocols including TLS, or something else entirely different. But until then, the unikernel infrastructure lets us solve the immediate systems problem at hand without letting that design decision vomit all over the rest of the application stack and burden us with a ton of technical debt.
> It sounds like MirageOS was only used because it was a reasonably portable network stack
It also happened to be the one that many of the team had authored, and written in a high-level enough style that we knew how to invert it. Nothing dramatic here, just picking the shortest solution to get shipping software that solved real user problems (in this case, VPN and firewall support on OSX/Win).
I expect that pulling off the same trick using several other user-level network stacks is also not hugely difficult, but I haven't used lwIP for some years to find out. The Rump Kernel NetBSD stack is probably the best alternative to start with, or the HalVM TCP/IP stack. The IncludeOS C++ one is also rapidly becoming a mature candidate.
Interesting, I was looking into running single daemons (e.g.: nginx) as a unikernel (through rump kernels). This is certainly something different!
Edit: doing some research it simply seems that Hyperkit/MirageOS are supplementing docker (via Alpine Linux). I may still be missing something just by looking at the slides.
This talk was more of a roundup of interesting uses of library hypervisors. The two are:
- the HyperKit/MirageOS use in Docker for Mac and how it can augment a native client application
- the far more general new Solo5 backend that IBM/Docker/Cambridge have contributed to MirageOS3, which shows how to use a library hypervisor to build a very Unix-like unikernel experience. MirageOS developers have also taken advantage of the libraries to add support for FreeBSD/bhyve very easily [1], so these new backends and libraries are increasing MirageOS library portability quite a bit.
[1] https://github.com/Solo5/solo5/issues/61
(I've just come back from Berlin and Docker Summit and have a ton of interesting questions and FAQs from attendees there. Will try to write up a blog post expanding on this presentation ahead of the MirageOS3 beta!)
The blog post is at: http://canopy.mirage.io/Liveblog/MirageOSUnikernelsDDS2016