It's not like evolution evolves towards the ideal, it just evolves away from the deadly. So unless you can show where external testicles will cause a group of people to die off, there's no reason to believe it should have been absorbed into the body. Plus for the species to continue, you don't really need that many males, anyway, so sub-optimal oddities involving men's bodies are likely to be more prevalent as it won't likely kill-off the species.