This line of thinking becomes a shield for executives, letting them push subordinates into corners where they'll have very strong incentives to do illegal things. And they will benefit from these illegal actions, while being responsible of nothing as they won't know enough on any subject.
This feels like a moral hazard paradise to me.
That's besides the point. If you want to operate in a certain country, you need to - as a business - familiarize yourself with their laws.
The CEO is ultimately responsible for ensuring the company abides the law. If the CEO doesn't know enough about French taxation, that's not an excuse not to follow the rules: the CEO should then delegate the task to someone they trust, but ultimately it's their responsibility.
That's usually the case for most crimes, which is why we have detectives investigate them.
How do you find a murderer in a large city? You Investigate!
Same goes for corporate crimes. Investigate!
Second: Cases like this often arise from different interpretation of laws that are all at least plausible. Companies may tend to be somewhat too optimistic in their interpretations, but it's rare, at least at that size, that they actively try to commit tax fraud.
Third: It is the company that profits from the offense, so it's only fitting to apply any sanctions to it.
Fourth: Making this criminal law would raise the required standards of evidence to a level where few of these cases could ever be prosecuted.