News for hackers. Stuff that matters.
Re:Docker 2.0 went from $11M to $135M in 2 years by kdrag0n on Friday January 13, 2023 @04:32PM (#34374176)
Looks like there's a lot of discussion about Docker Desktop alternatives here, so shameless plug: I've been working on a new Linux+Docker+Kubernetes solution for macOS recently! Already has quite a few improvements over existing apps including Docker Desktop, Rancher, Colima, etc:

- Fast networking: 30 Gbps! vs. 150 Mbps with Docker VPNKit. Full VPN compatibility, IPv6, ping, ICMP and UDP traceroute, and half-open TCP connections. (Future work: transparent proxies)

- Bidirectional filesystem sharing: fast VirtioFS to access macOS from Linux, but there's also access to the Linux filesystem from macOS. This can help with performance: for example, you could store code in Linux and edit it from macOS with VS Code (which can take the performance hit of sharing), so the container runs with native FS speed.

- Not limited to Docker or Kubernetes. You can run multiple full Linux distros as system containers (like WSL) so they share resources.

- Fast x86 emulation with Rosetta

- Bidirectional CLI integration like WSL

- Much lower background CPU usage. Only ~0.05% CPU usage and 2-5 idle wakeups per second — less than most apps, while Docker wakes up ~120 times per second. Made possible with low-level kernel optimizations. Also, no Electron!

- Better solutions to other problems that can occur on macOS: clock drift is corrected monotonically, dynamic disk size, and more I'm working on now. Will look into memory usage too, although I can't guarantee a good fix for that.

- No root needed

Planning to release it as a paid app later this month. Not OSS, but I think the value proposition is pretty good and there will be a free trial. Not sure about pricing yet. (Let me know if you have any thoughts on this!)

If anyone is interested, drop me an email (see bio) and I'll let you know when this is ready for testing, likely within a week or two at most :)

Also, feel free to ask questions here or let me know if there are other warts you'd like to see fixed.

Re:Docker 2.0 went from $11M to $135M in 2 years by babelfish on Friday January 13, 2023 @04:34PM (#34374198)
How do you achieve 30Gbps?
Re:Docker 2.0 went from $11M to $135M in 2 years by kdrag0n on Friday January 13, 2023 @05:20PM (#34374709)
Great question! I wrote a new userspace network proxy/stack in Go, similar to Docker's VPNKit and built it with performance in mind at all levels.

What makes it fast is support for modern NIC features that improve performance significantly, similar to those supported by Apple's in-kernel NAT (vmnet) but implemented in userspace. I've made changes to the guest kernel to implement these while working around limitations in Apple's Virtualization.framework. I'm not actually sure why it's slightly faster than vmnet in the host-to-guest direction (30 vs. 25 Gbps), but I'll take it.

Some snapshots of my journey working on the network stack:

https://twitter.com/kdrag0n/status/1606461436863352832

https://twitter.com/kdrag0n/status/1604288427306160128

https://twitter.com/kdrag0n/status/1607236475715989506

https://twitter.com/kdrag0n/status/1609013653214474240

Re:Docker 2.0 went from $11M to $135M in 2 years by zy0n911 on Friday January 13, 2023 @09:24PM (#34376739)
Have you got a GitHub link to this? Would love to take a peek.
Re:Docker 2.0 went from $11M to $135M in 2 years by kdrag0n on Friday January 13, 2023 @11:20PM (#34377366)
Not planning to open-source it, sorry.
Re:Docker 2.0 went from $11M to $135M in 2 years by ilyt on Friday January 13, 2023 @05:29PM (#34374786)
I'm more interested how docker achieves 150Mbit, that's dog slow
Re:Docker 2.0 went from $11M to $135M in 2 years by uberduper on Friday January 13, 2023 @05:43PM (#34374935)
I haven't used a mac for quite some time and when I did, I used docker-machine. I recall from my use and from trying to help others that were using docker desktop, it was unusually difficult to make your ssh-agent available in a container.

If that's still an issue, then please figure out a way to make that seamless.

Re:Docker 2.0 went from $11M to $135M in 2 years by kdrag0n on Friday January 13, 2023 @06:31PM (#34375371)
I have seamless ssh agent forwarding for full Linux distros, so if you're running Docker from a Linux shell (like you would in WSL), then the usual way (https://gist.github.com/d11wtq/8699521) should just work. Making it work when running Docker directly from macOS will be challenging, but I'll dig into it and see what's possible. Thanks for bringing it up!
Re:Docker 2.0 went from $11M to $135M in 2 years by uberduper on Saturday January 14, 2023 @01:05PM (#34382346)
Yea I use a linux workstation these days and wrote some tools that worked fine with docker on linux but was a nightmare to make work on any of the docker on mac solutions my coworkers were using.
Re:Docker 2.0 went from $11M to $135M in 2 years by turtleyacht on Friday January 13, 2023 @11:11PM (#34377325)
May I ask, what language(s)/stack is it written in?

It sounds like it has a ton of great features.

I guess virtualization is its own domain, much like video encoders, cryptography, and systems programming.

Feels like the gateway to those is C and assembly.

Re:Docker 2.0 went from $11M to $135M in 2 years by kdrag0n on Friday January 13, 2023 @11:28PM (#34377401)
It's a mix of Go, Rust, Swift, and C, but I didn't write any virtualization code—it's all built around Apple's Virtualization.framework because that's the only way to use Rosetta in VMs. (The architecture bits are behind a private API in Hypervisor.framework.)
Re:Docker 2.0 went from $11M to $135M in 2 years by turtleyacht on Friday January 13, 2023 @11:57PM (#34377498)
That's awesome, combining the strengths of (or hacking around) multiple languages.

Recent reads include Mac OS on Docker, rooted John Deere tractors, jart's Blink work, and now this :)

Re:Docker 2.0 went from $11M to $135M in 2 years by wslcorsair on Monday January 16, 2023 @02:55AM (#34398060)
As a fan of everything containers and WSL (what else), I'll be looking forward to see any demo and docs of your project. I understand it's not OSS and that's fine if it's your choice, but that's why your communication and demos will be key for the success of your app.

Claims are good, proofs are key (and I'm curious how it will separate itself from LIMA :)

I sincerely wish you all the best and as said, really looking forward to the first set of demos (videos/streams)

Re:Docker 2.0 went from $11M to $135M in 2 years by kdrag0n on Tuesday January 17, 2023 @02:27PM (#34417387)
Thanks! I'm working on it and I hope it'll live up to your expectations :)