Counter-example: http://jonathanwhiting.com/games/knossu/ (Highly recommended if you have 20 minutes to spare.) The geometry of this game is unlike any I have seen so far. There is common logic with that of a Doom-like ray caster, but I'd argue not much. The time spent rewriting the generic parts of a ray caster probably pales in comparison to the specific parts of his graphics engine.
(Of course, your point stands in general. But for Jonathan Whiting in particular, I have the feeling that it may not.)
As far as initializing the hardware, that's a relatively small, one-off piece of code that's different for every platform (including consoles) anyway.
This leads to a situation where copy-paste at the start of the project may be the best way to reuse good stuff. Otherwise too many assumptions change. If you try to make it a parametric problem, you usually just find out you introduced accidental coupling later.
i feel like this is somehow perpetuated like folklore, but there shouldn't be any reason why game code should be inherently less reusable than other areas of software development. For example, code that deal with geometry data shouldn't really be any different for games, or code that deal with setting up a rendering environment ought to be the same for most, if not all games (i can't imagine initializing directx or opengl would be different for different games).
Game logic is one area that is going to change a lot between games. However, if you partitioned your game logic well, wouldn't it make the next game easier since you're "just" swapping game logic?