News for hackers. Stuff that matters.
Re:Improving Docker with Unikernels: Introducing HyperKit, V... by kevinmgranger on Wednesday May 18, 2016 @11:21AM (#11722571)
docker's go-9p now makes for the 3rd implementation of 9p in go:

docker/go-9p https://github.com/docker/go-p9p

rminnich/ninep: https://github.com/rminnich/ninep

rminnich/go9p: https://github.com/rminnich/go9p

There's also the Andrey Mirtchovski and Latchesar Ionkov implementation of go9p, but all I can find is a dead Google Code link from here: http://9p.cat-v.org/implementations

Re:Improving Docker with Unikernels: Introducing HyperKit, V... by justincormack on Wednesday May 18, 2016 @02:45PM (#11724610)
We did try the other implementations first, but had a lot of issues doing what we wanted with them.
Re:Improving Docker with Unikernels: Introducing HyperKit, V... by pjmlp on Wednesday May 18, 2016 @10:47AM (#11722338)
With lots of OCaml love it seems, from a quick glance through the source repositories.
Re:Improving Docker with Unikernels: Introducing HyperKit, V... by danieldk on Wednesday May 18, 2016 @10:50AM (#11722360)
And Mirage. This release is really awesome:

- It puts Mirage in use in real production systems.

- It strengthens the ecosystem around Hypervisor.framework. After xhyve and Veertu, this is really nice. So much for the meme that newer OS X versions only integrate iOS features.

Re:Improving Docker with Unikernels: Introducing HyperKit, V... by floatboth on Wednesday May 18, 2016 @11:39AM (#11722702)
They only took, like, the network stack from Mirage. Their virtualization library is xhyve… well, turned into a library.
Re:Improving Docker with Unikernels: Introducing HyperKit, V... by avsm on Wednesday May 18, 2016 @11:44AM (#11722748)
Open sourcing our libraries is obviously just the start. We're really keen to hear about any other uses that people might have and move HyperKit in the direction of easy integration with higher-level frameworks.

For instance, direct ports of unikernel projects to run against the HyperKit kexec APIs would be really fun. Any takers for MirageOS or HalVM? :-)

We've started keeping a list of "help wanted" issues for anyone interested in getting started with hacking in this area: https://github.com/docker/hyperkit/issues?q=is%3Aissue+is%3A...

Re:Improving Docker with Unikernels: Introducing HyperKit, V... by bboreham on Thursday May 19, 2016 @06:06AM (#11729096)
What was the motivation to create a new repo with a different name rather than maintaining a fork of xhyve?
Re:Improving Docker with Unikernels: Introducing HyperKit, V... by avsm on Thursday May 19, 2016 @01:05PM (#11731694)
We're taking it in a fairly different direction from upstream xhyve and bhyve, with the integration with other kits like VPNKit, DataKit (and soon, FSKit). We wanted to be able to diverge the build system in particular, and this makes it difficult to maintain a direct fork.

However, we're committed to upstreaming patches to their relevant projects where relevant, and so the HyperKit Git repository is as close as we can make it. If it diverges a lot in the future with file renames, we'll have to figure out the Git-fu to make cherry-picks easier...

Re:Improving Docker with Unikernels: Introducing HyperKit, V... by geodel on Wednesday May 18, 2016 @11:10AM (#11722493)
> ...steering committee chair of the Commercial Uses of Functional Programming conference

From Anil's profile on blog and considering he is founder of MirageOS written in OCaml, it would be surprising if it were anything else than OCaml.

Re:Improving Docker with Unikernels: Introducing HyperKit, V... by tachion on Wednesday May 18, 2016 @11:21AM (#11722570)
I wonder if we'll see a move towards getting Docker working on FreeBSD using either Jails or bhyve finally, since it talks about using bhyve hypervisor... That would be really great.
Re:Improving Docker with Unikernels: Introducing HyperKit, V... by floatboth on Wednesday May 18, 2016 @11:38AM (#11722688)
There is a port of docker that works with jails+ZFS (and the linux.ko syscall translator to run linux images) https://wiki.freebsd.org/Docker

There's also jetpack for the app container spec (rkt) https://github.com/3ofcoins/jetpack

I'm also experimenting with building a simpler system.

Re:Improving Docker with Unikernels: Introducing HyperKit, V... by avsm on Wednesday May 18, 2016 @11:48AM (#11722765)
It should be possible to build a simpler port for FreeBSD with the recent Docker 1.11 release. It moved the container execution to containerd (https://github.com/docker/containerd), so that's where a bunch of the Jails logic would go.

I'm quite keen to see Docker running on FreeBSD so I can use it for my CI pipelines. I'm less interested in Linux emulation to run Linux images -- I'd like Docker support on FreeBSD to run FreeBSD images first!

Re:Improving Docker with Unikernels: Introducing HyperKit, V... by voltagex_ on Thursday May 19, 2016 @12:21AM (#11727940)
What advantages would Docker give you over jails themselves?
Re:Improving Docker with Unikernels: Introducing HyperKit, V... by evgen on Thursday May 19, 2016 @03:03AM (#11728554)
Can't speak for anyone else, but while I think jails are "better" for a variety of reasons it seems clear there is far more market momentum behind Docker. More particularly, its toolchain. Given the choice between building myself (or relying upon a small, but talented, pool of devs) and drafting off tens of millions in VC cash and a much larger community I think some would prefer the ability to have a *BSD CI chain that is just a config setting in a Docker setup.
Re:Improving Docker with Unikernels: Introducing HyperKit, V... by avsm on Thursday May 19, 2016 @01:07PM (#11731705)
Containerd is just one part of what Docker does. There is also image management (signing via Notary), distribution (pull/push to the Hub), orchestration (Swarm and friends), linked services (Compose), etc. You could reuse all the rest of it while running containers in FreeBSD jails as the method of containment.
Re:Improving Docker with Unikernels: Introducing HyperKit, V... by kordless on Wednesday May 18, 2016 @10:45AM (#11722311)
Seems like only a year ago Docker changed how it used Virtualbox to boot VMs using machine (and caused me endless amounts of suffering trying to figure out how to fix it). Now it would seem they are getting rid of Virtualbox entirely with their own VM...which needs contributions.
Re:Improving Docker with Unikernels: Introducing HyperKit, V... by danieldk on Wednesday May 18, 2016 @10:52AM (#11722375)
getting rid of Virtualbox entirely with their own VM...

Not really. This builds heavily upon Hypervisor.framework, which is included in new OS X versions. Hypervisor.framework is pretty cool, because it allows for building VM applications that are sandboxed and don't require kernel modules. As a result, Hypervisor.framework applications can even be distributed via the App Store (see e.g. Veertu).

Re:Improving Docker with Unikernels: Introducing HyperKit, V... by fpoling on Wednesday May 18, 2016 @04:18PM (#11725522)
The drawback is that now VM manager is at mercy of what Hypervisor.framework provides. For example, Veertu does not support redirection of USB devices into VM because the framework does not support it. That makes it rather useless for my usage and I stick with VirtualBox for now.

Of cause, this is not an issue for Docker as the framework covers all its needs.

Re:Improving Docker with Unikernels: Introducing HyperKit, V... by lobster_johnson on Wednesday May 18, 2016 @02:51PM (#11724664)
I wonder if Vagrant intends to provide a Hypervisor.framework-based Linux backend. That would be cool.

Edit: Experimental plugin: https://github.com/oldpatricka/vagrant-xhyve.

Re:Improving Docker with Unikernels: Introducing HyperKit, V... by rhinoceraptor on Wednesday May 18, 2016 @11:26AM (#11722601)
I'm running the new beta native Docker for Mac, it is really nice. You install from the DMG, enter your password, and you're done.

No more running 'eval $(docker-machine env dev)', no more having to delete the VM and rebuild it, etc.

Re:Improving Docker with Unikernels: Introducing HyperKit, V... by knz on Wednesday May 18, 2016 @01:02PM (#11723455)
Have your encountered any major bugs with Docker for OSX? I've been running docker-machine without any major issues for months but have noticed in the last few weeks that I need to reboot the VM lately due to weird caching issues.
Re:Improving Docker with Unikernels: Introducing HyperKit, V... by MehdiHK on Wednesday May 18, 2016 @08:38PM (#11727147)
Re:Improving Docker with Unikernels: Introducing HyperKit, V... by jacobr1 on Wednesday May 18, 2016 @09:53PM (#11727395)
Mounting sockets as volumes no longer works. This broke a few things I was doing with sibling builds (-v /var/run/docker.sock:/var/run/docker.sock)
Re:Improving Docker with Unikernels: Introducing HyperKit, V... by rhinoceraptor on Wednesday May 18, 2016 @01:47PM (#11723989)
I had it crash a couple times when it printed lots of text from the container to the terminal. I haven't had that happen since the update from today, though.
Re:Improving Docker with Unikernels: Introducing HyperKit, V... by knz on Wednesday May 18, 2016 @02:09PM (#11724240)
Thanks for the feedback.
Re:Improving Docker with Unikernels: Introducing HyperKit, V... by toomuchtodo on Wednesday May 18, 2016 @01:44PM (#11723958)
I have constant connection problems between the Docker CLI and the Docker VM running Docker for OSX :/
Re:Improving Docker with Unikernels: Introducing HyperKit, V... by justincormack on Wednesday May 18, 2016 @02:45PM (#11724618)
In recent updates? We have changed how this works in the last few releases, we hope it will be more stable.
Re:Improving Docker with Unikernels: Introducing HyperKit, V... by zwischenzug on Wednesday May 18, 2016 @03:31PM (#11725061)
I had this also; will re-try a more recent version.
Re:Improving Docker with Unikernels: Introducing HyperKit, V... by avsm on Thursday May 19, 2016 @01:07PM (#11731707)
Please do let us know if it persists, to [email protected] or on the forums.
Re:Improving Docker with Unikernels: Introducing HyperKit, V... by bonquesha99 on Wednesday May 18, 2016 @03:17PM (#11724908)
Wow that sounds awesome! Are there any estimates of when this will be an open beta for everyone? Very excited to try it out!

Docker ID: shuber

Re:Improving Docker with Unikernels: Introducing HyperKit, V... by rhinoceraptor on Wednesday May 18, 2016 @03:48PM (#11725248)
I'm not sure, I was on the waiting list for a while, I've been in the beta for about two weeks now.
Re:Improving Docker with Unikernels: Introducing HyperKit, V... by amackera on Wednesday May 18, 2016 @01:00PM (#11723432)
Docker is still pretty bleeding edge, and was never designed to work with Mac OS X or Windows from the get-go.

The fact that the proposed experience for OS X and Windows is so good is a testament to how hard the community and the Docker team have been working! It's natural that there would be some growing pains.

Re:Improving Docker with Unikernels: Introducing HyperKit, V... by jcoffland on Wednesday May 18, 2016 @01:10PM (#11723534)
So you're saying it's really bad and that's why it's so good.
Re:Improving Docker with Unikernels: Introducing HyperKit, V... by avita1 on Wednesday May 18, 2016 @10:50AM (#11722363)
My understanding is not great, but it looks like they're bypassing a host VM entirely and using a shim (HyperKit) to talk to whatever virtualization the underlying OS supplies.
Re:Improving Docker with Unikernels: Introducing HyperKit, V... by chuhnk on Wednesday May 18, 2016 @05:15PM (#11726035)
Very interesting work. I find go-9p quite fascinating and think it could really have broader applications. Docker if you see this, I actually think you're on to something for microservice development thats native to the docker world. I've been trying to come up with ways of replicating the unix philosophy around programs that do one thing well and the use of pipes but was always limited in my thinking in terms of http, json, etc, etc.

My advice, as a guy who's currently building something in the microservice space, explore this further. Spend some time building fit for purpose apps with this and see where it goes.

Re:Improving Docker with Unikernels: Introducing HyperKit, V... by andrew_wc_brown on Wednesday May 18, 2016 @10:45AM (#11722316)
I guess I just want to know the take away. eg. Will consume less memory on mac.
Re:Improving Docker with Unikernels: Introducing HyperKit, V... by amirmc on Wednesday May 18, 2016 @10:51AM (#11722371)
If you're asking about Docker for Mac/Windows, an earlier blog post has a few details.

https://blog.docker.com/2016/03/docker-for-mac-windows-beta/

In essence, it's a much nicer experience using the app and since it's improving all the time (still in beta, fwiw).