So the main thing Apple could do to show some love to docker is build out full apfs support in the Linux kernel. I have no clue how much work that entails but presumably it’s pretty massive, and it seems totally unlike them. Maybe one day they’ll have a come to Jesus moment like Microsoft and start caring about developers (non-iOS developers) but I don’t really see it happening.
Have you considered storing the npm_modules and running webpack on a tmpfs/volume (so that it doesn't have to go through the shared FS layers) and only copying the end product to a shared volume?
Don't get me wrong. They were good workaround years ago. But I hope we can do better.
(The efficient thing about this class of solutions, if you’re wondering, is that the client mounting the loop-image ends up owning + managing the loop-image’s internal filesystem’s metadata within its own local disk cache, such that it can coalesce filesystem metadata writes and only push a new copy of the disk-image blocks backing the filesystem metadata after a potentially-huge number of changes. With a network/host-guest filesystem, meanwhile, every filesystem metadata change must become its own synchronous message to the host, to be pushed to the host’s filesystem driver for linearization, so it can succeed or fail relative to other things going on within the host.)
The newer Docker for Mac is better, but file system perf still could be massively improved. Big node_module directories can still cause pain even today
is there any other kind?
(on service definition): volumes: - .:/usr/src/app:cached - node_modules:/usr/src/app/node_modules/ - next_artifacts:/usr/src/app/.next/
and then in the top level volumes key defining node_modules and next_artifacts as blank/default.
That means I mount everything except node modules and the build artifacts so the shared filesystem does a LOT less work trying to sync stuff. The downside, of course, is that I need to run npm commands both inside the container and outside if i want them in my IDE. A fair trade for decent performance. That setup is still not as fast as native but definitely usable and does not send my machine into space header mode much more than normal usage.
After eventually falling in love with it on Windows and Linux, I later tried it on MacOS... oh my. Not fun!
I was developing a database as a personal project that involves mmapping a file and the Docker shared filesystem had some peculiar behavior they was different from how normal filesystems work.
So much Unixness on the Mac OS boot ROM, the Pascal API, the lack of memory protection, and so on...
I used starting fresh on the M1 as an excuse to give MacPorts a go, and I like it much better than Homebrew. There's some smaller packages that aren't on MP, but all the big stuff is there and to me it feels much more like a Linux package manager.
Over time I'm not sure brew is any better now, though. I have unlinked versions of gcc, python, etc., etc. under /usr/local that are there just to handle brew packages that listed them as pinned dependencies. It's nice that brew doesn't expose them to CLI unless I want it to do so, but it's not less complex.
Assuming MacPorts has a good "bottle" type concept of precompiled packages now too (haven't looked for years) it's probably about the same as brew now, just more stable. If they still compile from source every time a la BSD, that would be my main sticking point.
https://saagarjha.com/blog/2019/04/26/thoughts-on-macos-pack...
Anyone tried Nix? I'm trying it soon. If it doesn't stick, yep, back to MacPorts like it's 2007.
Have there been some updates recently? About a year ago we were trying to use Docker on a windows host at work, and dealing with things like file system paths was a nightmare
WSL is really remarkable.
Technically there shouldn’t be file system issues right?
And not all are minor. Not being be able to run LXC is a deal breaker for me.
(disclaimer: not directly involved with WSL, just trying to help)
https://docs.microsoft.com/en-us/windows/wsl/compare-version...
https://devblogs.microsoft.com/commandline/access-linux-file...
https://community.openbiox.org/d/72-windows-subsystem-for-li...
You do have the occasional issues regards line endings; that's the only negative I've had so far.
EDIT: H12 is more accurate. The Linux OS is separate, but the Windows FS is mounted to a logical place and "just works".
I don't see how it's useful to say that it's a "modified kernel" due to it being packaged for a specific application in a way that is necessary to use it.
I like having the system strictly separate from my crap, and I think the UI is fairly good. The variety of packages available out-of-the-box is outstanding. I miss it when I'm on Linux, now, in a workstation-not-server context (yes, I know, there's LinuxBrew, but the package set is much smaller and less well-maintained). I started on MacPorts but got sick of it borking itself every few months such that it was faster to nuke the directory and reinstall everything than to figure out what it'd screwed up this time (granted, that was about a decade ago, maybe it's great now).
Brew gives off all kinds of signals of being something I'd hate (cutesy; a system tool written in Ruby; breaks with norms) but I like it a ton.
I've never used linux as a dev machine, but in general apt seems much more reasonable in this regard. My frustration with Brew is pushing me to seriously consider a linux machine, so if anybody has counterpoints here I'm definitely interested in hearing about your experiences.
I don't care that Brew "breaks with norms" I just care that it breaks my shit.
So I end up with:
System -> Apple-managed
My tools, as in programs I personally use -> pretty much entirely Brew, in fact I think on my current workstation this category is 100% brew-installed
Dependencies of anything I'm working on -> some language-specific version manager (which itself is may be brew-managed, actually) plus containers or VMs with scripted installs, probably.
On linux my experience is typically more like:
System -> package manager
My tools -> package manager, plus some sketchy extra repos that I hate to add but do anyway because I don't want to screw with manually updating things, plus several things installed manually, plus a bunch of things on older versions than I'd like but not worth the trouble/risk of finding some way to upgrade without it being a PITA.
Dependencies of anything I'm working on -> some language-specific version manager (almost certainly not available in the distro's official repos) plus containers or VMs with scripted installs, probably.
So for my use, Brew cleans up the "My Tools" workflow very nicely compared with Linux, excepting, kind of, my days back on Portage/Gentoo, which of course has its own problems.
Only difference I see is that my containers are running natively (eg no VM in the background) and that I've not had any random errors from my package manager in years. Not sure what brew is like these days but last I used it the experience felt like a half baked apt/yum to me (3+ years ago though)
For me, Brew is for managing my personal software I use that doesn't come from Apple. Project dependencies, including the version of the compiler or interpreter for the language you're writing, don't belong brew-managed in most cases, which seems to be what's tripping people up when they try to use it for that.
Yes, containers run better on Linux because they're native. No quibble there. I just find I'm much, much better able to cleanly manage my personal software (not project dependencies, which, again, I wouldn't try to manage with my workstation's Linux package manager, either) with Brew on macOS than in any Linux distro I've used. 99% of what I ever want to run (outside the base OS, and project dependencies) is on there, available at a single "brew install", after I do nothing more than install Brew itself, versus 50-95% on Linux (depending on the distro), where I find myself adding all kinds of extra repos and installing one-offs a variety of ways just to get to a baseline level of having all the stuff I need at new-enough versions. And the interface is above-average, in my opinion (but again, Portage/Emerge is my favorite package manager on Linux and maybe the only one aside from Void's that I've found pleasant to use, so I may just be weird)
Anything I need at a particular version gets installed some other way, unlike how I operate on Linux, where anything I need at a particular version plus a bunch of other stuff all gets installed outside the package manager (talking workstation usage specifically)
https://www.postgresql.org/download/macosx/
Likewise Python and other programming tools where running a specific version is important are best managed outside homebrew.
apt and yum don't go doing things you didn't tell them to do (usually)
Tangent warning:
When WSL2 came out I ended up moving away from WSL and moved to running my development environment inside a VMWare VM and connecting to it via VSCode remote ssh development feature (the same one used for WSL). This is essentially a manual version of WSL2, something people have been doing for ages.
Now, whether I develop from Windows or MacOS, I am always connecting to my Linux VM. If I need to run docker, it's running in that VM.
I got to the point now where I forward my SSH port from my public IP to my desktop PC at home. When I am away from home (e.g. working in the office with my laptop), I connect to my desktop (via ssh on my public IP) and due to the seamless integration VSCode has for remote development, It feels like everything is running on my local device.
The PC is of course handling compilation, intellisense and all that good stuff while my laptop is essentially a thin client. The laptop now runs faster, has a longer battery life, doesn't spin up the fans and also doesn't need to be upgraded.
My home PC, electricity and internet bills are also tax deductible because I use them for work purposes.
When you say "proper Linux distribution", I assume that's still CLI only? Do you develop with e.g. emacs, vim on WSL, or do you have some IDE with remote running and debugging into WSL? Or have a missed a trick and in fact X/Wayland applications can be run on WSL?
With openssh in windows, it's been quite easy to run an x server in windows, and use ssh with x forward for gui access.
But rdp feels more native to windows, and both x and Wayland has rdp server backends - so you can do things like: https://www.nextofwindows.com/how-to-enable-wsl2-ubuntu-gui-...
But apparently ms is working on a Wayland compositor allowing directly running gui apps - I don't think it's quite there yet:
https://www.phoronix.com/scan.php?page=news_item&px=Microsof...
https://github.com/Microsoft/WSL/issues/938#issuecomment-763...
Somewhat related: https://ltsp.org/ I'm not sure about the state of a non-x rdp server for thin clients though.
From the look of it, if I were to set up the WSL2 Ubuntu GUI with RDP, that'd tick all the boxes, right? And as a bonus I'd be able to access files in WSL2 from Windows.
Any idea how WSL2 interacts with VPNs? I've seen some mixed reports around the web but this is also a must-have for me, if I can't use a VPN in my dev environment it's game over.
P.S. thanks for making an account just for this reply :)
Fwiw wsl2 runs in a vm now (hyper-v). AFAIK it's in order to increase fs performance "inside" Linux (so faster compiles if compiling in wsl).
It's also possible to use cifs/samba shares - but I don't know what you're compiling? Chrome/Firefox sized c++ projects?
Might be worth it to try with a tmpfs/ramdisk either way?
No. WSL2 uses it's own, minimal (proprietary) init, which basically launches default shell for the configured user and that's it. No service management in sight, and no equivalent of systemd's user scope or user session either.
That said, I've been playing around more and more with WSL2 on a secondary machine as a current primary Fedora user (for about 10 years now) and I really haven't found a good reason why I would need systemd in the WSL2 VM vs the custom init.
In practice that hasn't been a roadblock for me, because VMWare Workstation 15.5 finally supports running on top of Hyper-V, so I can have both working at once. Moving everything to a single hypervisor API has some nice benefits...
/s, obviously.
Most WSL2 users use it as shell to run occasional ELF/x64 binary that runs in console; for services, they would use Docker Desktop for Windows. Anything beyond that and you will quickly find out, that it is not really a standard linux distribution.
https://blogs.windows.com/windows-insider/2021/01/06/announc...
Note this only impacts calling the Linux go toolchain from the Windows side (via the /$wsl/ path). Its not an issue inside WSL... but it comes up when you want to say configure Windows IntelliJ/GoLand to use the Go compiler hosted inside of the WSL VM.
> When you say "proper Linux distribution", I assume that's still CLI only? Do you develop with e.g. emacs, vim on WSL, or do you have some IDE with remote running and debugging into WSL? Or have a missed a trick and in fact X/Wayland applications can be run on WSL?
You can run X apps just fine in WSL2 as long as you have a display server running on the Windows side. I use X410. Important to note, Microsoft is working on native Wayland support long term. For what it is worth, I run IntelliJ/GoLand from inside of WSL and use X410 to render the GUI. This works great.
> I think my end goal is to have a contained development environment whilst not being forced to use specific tools (i.e. VSCode remote debugging, eugh) and not sacrificing compilation times by running in a VM that's too slow.
Are there any other gotchas to working like this? As it happens I'm a go dev using GoLand so your go-specific issues are of interest to me. I'm not too bothered about not being able to use Windows GoLand as I'd be just using it from WSL2 anyway, but I'd be interested to know if there are any other pain points. Any issues with VPNs, if you're using them?
Edit: Im switching from Fedora to a WSL setup for work in the next week. Ill report on issues I encounter in that migration. I suspect VPN will come up because one of the key reasons im switching off Fedora to Windows+WSL is due to corporate VPN requirements.
If you install an X server on Windows you can run graphical WSL apps. It runs really well too. Years ago I used to run Sublime Text straight from within WSL.
I've been using WSL / WSL 2 for a few years now for full time web development. A while back I made a video going over all the tools I use and how I have Docker, WSL 2 and a bunch of other things configured at: https://nickjanetakis.com/blog/a-linux-dev-environment-on-wi...
VScode directly connects to WSL. You can open folders/files with 'code <folder>' from the WSL terminal, save, run whatever from VScode etc.
As for Docker, you can also use the GUI from within Windows that automatically connects to WSL but admittedly I almost never use the GUI.
For example, when postgresql 12 was released, it took some months to appear in brew. Meanwhile you could not use alternate taps to resolve dependencies, if some package required postgres, it had to be the original one.
Those times are way gone, that's the purpose of containers.
I've been using Brew for a while to just install "core" packages like python, curl, wget and such, and everything else like a postgres, nginx, whatever..a go to a container.
Also, I have some tools installed with brew, that have postgres as dependency (e.g. pgloader or mapnik).
Please elaborate on the claim that "running a SQL database" is the purpose of containers.
However, if I do some exploratory experiments, where I don't care about repeability and where I use other local tools (like the mentioned mapnik, or jupyter), having it in container is needless complication.
By and large, there’s no such thing as a container, there’s just sprinkles of housekeeping magic. To wit, Docker implemented in around 100 lines of bash:
https://github.com/p8952/bocker
Problems come when we think that today’s containers manage to actually contain anything, bring any security guarantees, or do much else than just slightly-more-successfully jump start a configurable bundle of dependencies.
Why wouldn't you want to run a database under VT-x, with random emulated hardware and a dependency-bundled disk image? By and large there's no such thing as a VM, there's just sprinkles of housekeeping magic?
Containers as specced and implemented do come with security guarantees. And if they fail to meet them it's a bug.
- Homebrew is a general purpose package manager, and Postgres is a package you might want managed.
- If you're using Docker/Docker Compose for a project anyway, that's the obvious way to do it.
- Postgres.app is a specialized tool just for managing Postgres installs, so it's hard to beat if that's what you need.
Some thoughts on the tradeoffs though:
- Homebrew really doesn't like the idea of "versions". It wants everything to be on the latest. That can be fine if you just need a tool locally, but if you want dev and prod to match, it is a pain in the ass.
- Docker isn't really very good at persistence. That's probably not a problem for local development, but you should be aware of it. Running it on a Mac introduces speed and memory issues you wouldn't otherwise have. And now obviously there's the M1 problem.
- Postgres.app is another thing to install. If you just need Postgres for one particular project you might not know about it or want to deal with installing something new.
https://applehelpwriter.com/2018/03/21/how-homebrew-invites-...
https://askubuntu.com/questions/261326/is-it-safe-to-chown-u...
I had a lot of issues with Brew, but the biggest one was how slow it was. Upgrading all packages on my Mac used to take hours.
It’s also just so slow to update if it’s been more than like an hour since you last updated, the way it uses one big git repo under the hood is just chaos.
You can install specific versions, but it requires some gitfu - you need to uninstall, find the brew commit where the package is at the version you want, then install from that specific git blob.
What you're looking apt is apt-holding:
apt-mark hold libxfont1
That been around since 2013 IIRC. And there was dpkg way of doing it before.
If you don't run it daily, it takes about a minute or two to update.
But even when it doesn't update, it is extremely slow compared to any other package managers. It is disruptively slow and it takes a lot of resources, even in a powerful machine (and I'm not talking about compilation here).
I do like tools to be as fast as possible, though, and I’d forgotten about that update it does sometimes when I run it - that does seem to take a long time.
I’ll have a look at how it works and see what the things are that take time. I would expect network traffic for updates, perhaps whatever it’s getting updates from does some processing (I’m sure it said something about GitHub), perhaps there is some dependency resolution that needs CPU...
It would be interesting to compare its architecture to other package managers if they’re significantly faster.
99% of the time, installing a package consists of downloading a few zips from their CDN, decompressing and linking. For those cases Brew could just be checking an API instead of constantly cloning the git repository.
I'm quite surprised nobody has reimplemented it in Rust or Go. The architecture is quite simple compared to a normal package manager. Maybe it's just superstition: people see "package manager" and assume it's complicated instead of digging into the code and finding out how it works.
Installing anything with brew involving multiple dependencies is also taking forever-ish, compared to mere seconds with apt.
Seems to be some signing issue but it’s only happened to me twice and I didn’t have time to properly investigate at the time...
However, when I last spent a couple of sprints attempting to get them working side-by-side, it was a pretty big failure. I couldn't get VirtualBox 6.0 running without falling back to soft virtualization which was painfully slow (booting a Ubuntu box took the better part of an hour).
It's a planned initiative for the future, but it means a fundamental change in a number of tools and would require a rather large set of rewrites.
On top of that, out of the large pool of users, only a small handful would actually need to run Docker side-by-side with these tools, meaning that it's a huge rewrite to allow a small number of people to use Docker on their desktops. That ultimately means it's getting very little traction and keeps getting pushed down the backlog.
It is my hope that Docker continues to accelerate down its current path toward replacement by other systems such as podman.
I suspect that Apple has little motivation to add a Linux kernel alongside the existing BSD-compatible kernel, even though macOS does have a built-in hypervisor framework/API.
BSD containers (e.g. jails) seem like a better fit than Linux containers (e.g. docker, lxc) for a BSD-based system.
If I were Apple (ehem) I'd spend a small but meaningful budget supplying devs to projects like Docker or TF to help speedup M1 adoption. Given that the chip market is open for grabs I'd say it could give Apple a much stronger headstart with their in-house silicon strategy, even if that means helping improve products or the bottom-line for well-established corps, some of them competidors.
The M1 migration is less than a year old and it’s already the single best/ most successful CPU migration in my memory. Far better than Apple’s PPC -> Intel Migration and massively better than whatever half-steps Microsoft has done to port over to ARM.
Apple absolutely should invest in much of what you suggest. In 6-12 months, after the platform is complete and mature enough to identify where the big problems are.
Most of us understand that you exercise caution when migrating mission critical work onto a 6 month old platform.
In the mean time, Apple has been hiring top Docker talent like Michael Crosby.
https://www.protocol.com/apple-hires-cloud-open-source-engin...
It’s possible they are hiring top container developers just to improve their internal cloud infrastructure. But what kind of hardware do you expect these guys will be running?
EDIT: Trimemd some repetitive stuff.
Too bad because a native apple docker would be really really useful. imagine:
FROM macos:10.13.3
RUN xcode-build
BTW macOS being a BSD derivative of sorts might benefit from Docker development on FreeBSD, which uses jails instead of cgroups, etc. At least, that could be easier to port, assuming that macOS kernel has the needed facilities.
Call of Duty is a whole other problem.
Not if your Windows users have to use Direct Access to connect to your container registry...
Last time I checked, the terminal was horrible to work with.
No need for WSL.
Docker and Developer Tools does not happen to be one of them.
[0] https://www.reddit.com/r/blender/comments/jsc03l/blender_on_...
a) It doesn't play well with deep sleep mode, and crashes. Perhaps it's been fixed, I don't know, I just disabled deep sleep.
b) The networking is different than WSL1 and requires odd workarounds for things like X11 to work normally. I had to use a 192.168.1.x address for DISPLAY instead of localhost. Which required some VB scripting to reliably work for me.
Other that this I love Windows 10 + WSL2 as a dev environment.
https://code.visualstudio.com/blogs/2020/07/01/containers-ws...
I believe one of the issues laptop-related, and I work exclusively on a desktop these days, so that was a non-issue for me.