I don't know in what way this is some trivial "recall" problem as you suggest. Does the reversed implication in question need to be in the training data explicitly or not? If it does I don't understand how you can claim a logical inference capability.
No that's not what the paper says. It says training won't immediately store this information in a way as to make it reversible.
Let's get one thing straight. This is a common problem for human learning as well. Anyone who has used Anki for language learning will tell you that if you just train on target language word on the front and native language word on the back, you will fail the reverse unless you specifically train for that.
This is specifically a problem of recall. Not a problem of making the logical inference. If you ask the human language learner immediately he has learnt the new word for the reverse direction, he will obviously tell you the correct word.
But later he may not recall the reverse even if he remembers the original. Again this is a problem of recall rather than the ability to make logical inferences.
In the same vein, if you give the LLM the original direction in context and immediately ask the reverse, it will correctly tell you. It can make the logical inference.
And although it's a complete waste of time because the answer is obvious when you do do these types of adversarial experiments, like adversarial blockworld, you find that LLMs are not internalizing abstract principles inherent to reasoning and are instead doing something like string-pattern-patching.
This isn't surprising if you think about how the machine actually works rather than treating it like a sacred magical box.