Of course I suppose it still works in that case. You found the reason. The reason sucked. Therefore, remove it.
The fact that people might come up with silly/nonsensical/bad justifications for the fence doesn't seem undermine the importance of doing due diligence to determine if there is a good reason for the fence in the first place.
It follows that you can't remove a fence if you cant figure out why it was created in the first place.
Its very common in natural language to state general principles which apply most of the time without intending it to be an iron-clad rule that would apply in any circumstances without exception.
For example, if there is a "Stay off the grass" sign, you'd avoid going on the grass, but you'd still go on if the need was important enough.
The case of being unable to find the reason is an exceptional circumstance in which the principle does not apply, at least in quite the same way.