> Pentium 4 was absolutely the right design to take advantage of faster transistors
No, it was a horrible design in many ways, irrespective of those constraints and any wishful thinking about the power wall, and it gave the advantage to AMD for years. Not a big surprise as it was designed by Intel's B team. The Pentium M which has been the microarchitectural basis for their subsequent desktop processors was designed by the A team.
To be fair, nobody has ever designed a CPU based on current process technologies. Design teams are always saying "we think the process technology we'll end up working with will look something like this...", and based on past history there was no reason to think that the power issue wouldn't get solved like it had the past N times.
Intel was quite correct in predicting that we would continue to get faster transistors, and the Pentium 4 was absolutely the right design to take advantage of faster transistors. The story of the past decade of CPU architecture is all about heat becoming a primary design problem for the first time in history.