When I wrote some parallel code in C, enabling hyper-threading resulted only 30% increase in speed, which not too little, but you would expect more for doubling the thread count. I don't remember what was the bottleneck, but it was not I/O. On an other occasion a consultant for a HPC consultant advised us to disable some physical cores for the optimal performance of a geological simulator application, because the core/cache ratio is higher that way.

30% increase in performance from hyperthreading is impressive.

Like much of the advice "it depends" and "measure it".

Basically hyper-threading shares a whole load of resources between threads. If your code ends up contending on those resources you probably run slower. If your code is blocked due to lack of instruction-level parallelism, it probably runs faster. Generally an expert-optimized numerical code (e.g. machine learning) is going to fall into the first camp as it's pretty easy to kill the ILP bottlenecks in that sort of code.

No. It's a performance optimisation and generally makes things faster. The only reason to disable it is because of Spectre security stuff.

No. It's a performance optimization when there are more threads than physical cores because it helps hide memory latency behind threading.

Yes exactly. It's a performance optimisation that generally makes things faster.