Every scientific model and analogy is flawed in some way, but many are still useful.
By noticing that while they have significant differences, those structures also have significant similarities, we were able to correctly answer one of the greatest questions of all time: where did humans come from?
Of all models, this one is particularly useful.
As I have mentioned before these structures are very different, compare the giant spheres used by snakes, bird, frog etc to process visual information and try and map that to a rodent, bat, or human. Hint you’re not going to be able to locate an obvious sphere because the fundamental morphology is just different.
Going back to your analogy saying visual cortex of frog is wildly different from a human visual cortex. That's similar to saying a frog backbone is wildly different from a human backbone. Sure that's true but there's also a shared common core functionality largely conserved across time
So yes frogs and humans both have a visual cortex, because we both have eyes. It really doesn’t explain anything beyond that point.
Frogs, birds, etc have very distinct spherical visual cortexes. Humans and squirrels don’t have that kind of spherical structure. So, other than naming where vision is processed the same thing the overlap isn’t particularly strong even at the most basic structural level.
It’s perfectly reasonable to extrapolate from studied organisms to species that share recent common ancestors. But reptiles don’t represent the probability distribution, they represent different results from that distribution under different evolutionary pressures. Extrapolation backwards and our common ancestor was very different from all decedents.
Importantly modern reptiles and humans have experienced the same amount of evolution. There’s vaguely similar traits, but we don’t call whale flippers hands because that’s what the structures was in some of their ancestors or even what it is in related species. Similarly we don’t call human hands flippers because that’s what the structure started out as even further back and it’s still preforming that function in modern fish.
The human visual cortex for example is larger than most creatures entire brain, it’s a wildly different structure than you find on a frog.