It depends on a lot of factors, but if your CPU physically does not have multiple cores then you can be 100% sure that your threads will not be executing code literally in parallel.

I think on most operating systems, if you have a multicore CPU and very little load apart from your program, and you run CPU-bound code in multiple threads, then you will see those threads executing in parallel in different cores.

To build on what you're saying: while true from the user perspective, out-of-order execution and instruction level parallelism means there are some types of parallelism happening at the single core level. However, these systems are designed to produce results to the end user as though no micro-level parallelism is occurring.

I mention this mainly because I've recently discovered the joy of SIMD intrinsics. While it's pretty difficult to gain anything from out-of-order execution (cough other than a spectre) it's possible to take advantage of SIMD through compiler autovectorization, intrinsics, or assembly coding. SIMD doesn't have the problem of race conditions though as the parallel operations are non-conflicting and the user view of the computation is synchronous. I imagine under the hood there are all kinds of tricks at play for complex instructions that involve different microcircuits that take different numbers of cycles.

EDIT: clarified last sentence

SMT (Like Intel's hyperthreading) can actually run 2 different instructions from 2 different processes in the same clock cycle on a single core.

Are there any operating systems that let you explicitly run CPUs in parallel?

I'm thinking n threads where the process requests the operating system scheduler to start and stop them all at the same time. Of course the OS would be permitted to refuse if it wasn't capable (or just not allowed).

A synchronised cpu master-slave relationship could be beneficial to parallelize some of the middle ground between instruction level parallelism and multi-threading

You can do a set affinity on Linux but it isn't a guarantee.

Why would you want to override the scheduler like that? That would also mean not scheduling your process until all the CPUs were clear, and greatly increasing scheduler coordination overhead. So it would decrease overall work throughput.

CPU affinity is genuinely useful, but I can't see why you'd want this kind of temporal affinity. Especially since you could have different CPUs running at different speeds!

>Are there any operating systems that let you explicitly run CPUs in parallel?

That's a good question. I don't have the answer, to be sure, but I would think part of the problem is conflicting priorities and who would "win" in that scenario.

CUDAs in a GPU might be more worthwhile for that approach, maybe, but I could also be talking out of my arse, here...

You can get that effect by pinning processors to threads and you can do the sync yourself with a barrier (like Java CyclicBarrier, not a memory barrier.)