It's not necessarily that simple, but it is simple. You need to do two things:

1) Integrate the master branch (or whatever your guaranteed-good branch is) with the code you're about to push. This prevents integration conflicts from causing the build to fail.

2) Test it on a reference machine. This prevents environment assumptions from causing the build to fail. (Such as installing new software or setting an environment variable, but forgetting to make it part of the build.)

These are both easy to do. My preference is to push to a testing branch on the integration machine, merge in the master branch, run the tests, then merge the testing branch back into the master branch. (There's a bit more to it than that, to cover edge cases, but that's the gist.)

Sadly, most teams and CI tools aren't set up to do this--although, as the article says, it's not rocket science. In fact, I'm surprised it's not obvious that you should do it this way.

Doesn't help if eg you have multiple reference platforms - many open source projects support a range of platforms (eg OSX, Linux, Windows) - much harder than in house software that has a standard platform. You really have to stage commits then.

While both of those are nice, even doing something as dumb as having a git pre-commit hook that runs 'make' (or whatever is the equivalent) catches an amazing amount of errors.

This was mentioned as a solution in the article.

http://graydon2.dreamwidth.org/1597.html

I agree. I thought that this was the whole point of having a CI system.

I don't have a huge amount of experience in open source projects, so maybe they do it differently. Anywhere I've ever worked, not breaking the build and not causing regressions were a prerequisite to getting any pull request serviced. That's how you keep the master from breaking.

On the other hand, building a testing in a clean environment costs money. A company will pay for server time if they believe it's cheaper than developer time (it is). Maybe OS projects just don't have those resources.