> I find it to be an extremely sad state of affairs when it requires scouring the internet for a blog post from a Darwin engineer to build an outdated XNU :(

Large projects at companies tend to be hard to build, simply because it takes a lot of effort to make the build system for something so complex easy. If community contributions aren't particularly valued (which they aren't for XNU), then it's hard to internally justify spending the effort. Linux hackers are spoiled…

XNU isn't even one of the worst offenders. Try compiling Google's Skia, for example…

Really? I found Skia easier to compile than actually use.

One needs to duplicate Google's build infrastructure, which is quite a pain versus other open source projects that rely on more mainstream tooling.

Which would be fine if they provided some straight forward way to replicate that (like a container). It really seems like open source doesn't have that high of a standing at Google (anymore?). I guess limiting hardware manufacturers use of open source, not spending time making sure other people can build open source code, is what gives you the promotion these days

It's probably gotten better lately. A few years ago it took me several days to get it built.

So it is not only me.

On my case I ended up giving up and went back to Qt.

I also found this to be true when I tried to use Skia.

I don’t see why compiling XNU needs to be more complicated than 1. install the compiler (easy to do just by typing ‘cc’ on a pristine Mac) 2. Type ‘make’

It doesn't need to be that more complicated, but build systems inevitably have a way of needing at least one full time engineer to get them to the point of being that nice to use.

That’s not how software teams work unfortunately.

There’s probably three build systems and none of them work on certain days of the week and the guy who wrote it all left six months ago.

For XNU? Doubt it. It's probably all on XBS and has multiple people maintaining the build system.

Sorry I forgot the /s on the end of that.

> Heap buffer overflow in the network stack allowed for sending ICMP packets that would crash devices.

It's not receiving an ICMP packet that causes the crash, it's when sending an ICMP error response packet. ICMP error packets can be generated in response to any type of IP packet, for example "host unreachable" when using UDP. These packets copy the original header of the packet which generated the error, which is where the out-of-bounds copy is.

I understand why the author doesn't want to give full details, but in this case it leads to a lot of misunderstandings on how it works.

I understand it to be a classic buffer overflow using a memory copy. Bounds checking fails and a buffer is overflowed with bits from the attacker. The author was able to cause a kernel panic. This does not mean another attacker could not execute arbitrary code using this exploit.

I expect to see a PoC soon with arbitrary code execution in kernel space on all unpatched MacOS/iOS devices soon. Which would be bad...

Presumably, internally to Apple, XNU is built mostly on build-servers. Maybe even the branches having CI build-on-push, and test machines able to PXE-boot directly to a given branch's tip build. In such a case, you'd never need XCode to successfully build a standalone XNU at all.

Nah. What you say is partly true, but I and others build xnu locally and incrementally all the time. Fast enough.

If you want more details: Lots of parts are obviously pure C, but in my experience the C++ stuff is usually quite incremental as well. Changing a header included in lots of places is the obvious multiplier to build time.

Linking does take some time, but it’s only really noticeable if you do very incremental changes (e.g. change a single value, build, boot, repeat).

Overall, building a whole kernel from scratch even for multiple configurations is not unbearable during normal development.

Developers of course never wish to compile a 5 line patch locally and launch a kernel in a VM to test it.

That's something practically every linux kernel dev does, but surely every time a XNU dev wants to add a single debug print statement, they'll go through the whole CI system before being able to run it.

If XNU can't be built incrementally (and, not having built it myself, I'm unsure whether it can, since it contains a large amount of C++), then it's going to take just as long, if not longer, to build it on your workstation, compared to going through the CI pipeline (with the beefy build-boxes.) So there's no disadvantage in a CI-only workflow in such a case.

IIRC Chrome has this workflow for this reason.

Chromium absolutely does not only build that way.

I occasionally build chromium on my machine, and sure it takes two hours the first time, but incremental builds after that are only on the order of minutes.

C++ code can be incrementally compiled... Sure, it's not as good as in many other languages due to headers being so heavy, but it's still a massive reduction in total build time.

I promise you incremental builds are extremely important. A completely clean build of iOS/OSX years ago would take multiple days & I'm sure it's only gotten worse. The entire internal CI system relies on incremental builds working properly so that they can generate an internal iOS build every day for testing. It's possible it's disabled for certain projects but I doubt it. C++ in no ways alters the incremental build mechanism; all the problems of incremental builds it inherited from C's preprocessor. Speaking of, that's what C++ modules fixes (clang has support for C & C++ there although when I left only C projects had it on).

Apparently it was actually already fixed in ios 12.0 and macOS 10.14.0 but retroactively added to the security notes for those releases only just now.

Thanks, updated.

and yet I always hear the argument, "but it's open source, just like Linux".

It is open source, but not like Linux. These are more of source dumps than an actual window into the development process.