I think there is a big gap between the two claims -
1. Human visual cortex is very plastic
2. There is no inductive bias in the visual part of the brain that has been "trained" by millions of years of evolution.
I trust you are correct on the former, but the latter does not seem to follow from the former.
My point is that in contrast to fruit flies, humans (well, mammals) raised in environments with non-natural scene statistics show significant differences in coding in the visual cortex. Here are some random references I dredged up:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4210638/ https://www.sciencedirect.com/science/article/pii/S096098222...
It's not hard to see why this might be selected for in mammals. The historic range of D. melanogaster is just sub-saharan Africa, whereas the historic range of humans is huge, and we are primarily visual organisms. So it would be advantageous to have different tuning if you lived in the desert vs the forest. Of course you have to look at the range and divergence times of the full phylogeny but I am getting distracted from my actual research. It's a very interesting line of thinking though. I was quite surprised to learn things were hardcoded in flies when I joined a fly vision lab.
The extreme plasticity of the human visual cortex is widely documented (studies in blind people, etc).