The bicycle might be more fun to ride
It's like how people often make fewer mistakes with slide-rules or abacuses than pocket calculators, since the former two require you to check your answer against your intuition, while the latter just tells you "the answer" to your (potentially ill-formed) query. Trusting a black box can be quite dangerous.
More seriously though: if the oracle were perfect, you should have to do almost nothing (i.e. the relative time savings would be near 100%; e.g. creating pizza: the oracle, you just tell: "get me pizza", whereas with the bike, you have to bike there—which I admit may be fun)—so as long as you have things you enjoy doing more in your non-productive free time, then it shouldn't be an issue. (Not that I think that's necessarily likely...)
My first thought for getting around this was something like formal verification that a black box does what it says—but once 'what it says' is too complex for a person to really understand (e.g. a black box containing the behavior of a human brain), you've hit a wall with this approach.
The other thing I can imagine is building up 'bicycles' which have parts of proven-correct medium complexity, arranged together in a single higher-level system also of proven-correct medium complexity (where medium complexity is: more complex than bike parts, but still with readily human-comprehensible semantics). I'd characterize these systems as sort of like Iron Man suits. Maybe that doesn't really add anything of practical benefit beyond the original bicycle analogy though ;)