> If your using it very poorly, you'd see a massive 2~x benefit.

I find this statement tough to agree with. It depends on the kind of work your processors are doing: it is IO heavy work or are you running Math computations? There are other axes basically related to how much work can be done by the current thread while the other thread stalls waiting for data to be fetched (or other non-parallelizable dependencies to be available).

So if you're getting a high benefit you shouldn't necessarily feel embarrassed!