The folks who designed those old amber IBM terminals (that I still use at work) would have argued that:
- Black background is easy on the eyes.
- The flashing cursor draws your focus.
- The amazing bold font technology breaks up the page very nicely.
So it would appear that "good design" is just a function of the technology available.
It's not arbitrary however.
Here are some essentially inviolate principles of good design: http://graphicdesign.spokanefalls.edu/tutorials/process/gest...
For example, there's no reason why search boxes should have rounded corners - it's just an arbitrary style. But since users are familiar with rounded search boxes, using them could make your site easier to use.
That somewhat explains the trends you see in design, because when we employ a convergent design standard, it's more readily accessible in a single visit than a design that breaks away from current conventions.
It's also empirically the case that the convergent design standards represent the 'fittest'.
And I would argue that evolutionary speaking they represent 'good design' (that's my definition anyway).
White space is a fundamental of good design.
Light gradients and thin grey lines provide subtle distinctions between content areas without overpowering the content.
A strong, obvious 'Call to Action' button in a vibrant colour encourages click through action.
Sans serif fonts work better than serifs in websites because of the inherent limitations with rendering fine details using pixels (they often also communicate 'modern').
Moving secondary links to the footer is simply good UX. They're available but out of the way.
940px (I'd say 970 actually) is the current sweet spot if you want to target the majority of people's screen resolution to provide a good experience.
(I agree with HNer... the examples you provide all look very different, but are just fairly well designed)