I work on the repairs side of this industry (diagnostic tools), and have come to the conclusion that while the thermal expansion is an issue, it’s probably not the largest or most fundamental one.

When an engine idles (with no load), it produces relatively little heat. Comparatively, an engine running faster, under load produces much more heat. The engine will warm up more quickly and less evenly when loaded, likely leading to thermal stress, and possible mismatched thermal expansion. On the other hand, modern lubricants are very good at handling different temperatures.

The other thing to consider is that the engine is only one part of the vehicle, and various other components of the drivetrain may actually suffer more thermal stress from the idling than from immediate driving.