No, hardware bugs could work like you describe, but Spectre/Meltdown does not make the result of executing a program depend on the results of speculative execution. It just makes timings depend on such results. If you can't time things, which a deterministic program can't (access to any real timer makes a program nondeterministic), you can set up spectre attacks but you can't receive the results.
Only if you define 'deterministically' to mean 'determined only by the program's own legitimately accessible context'. These timing side-channel attacks work precisely because the processor performs in a deterministic and detectable manner, but does so based on information which the attacking program should not be able to access.