"Taking 50,000 international units (IU) a day of vitamin D for several months has been shown to cause toxicity" [1]
Regardless of that though, I agree that the UL is really difficult to quantify. Furthermore I'm not sure anyone could cite any article that shows a significant health benefit to supplementing above 10k vs below, hence, the rational choice would be to try stay below that.
However all of this talk about IU intake is secondary to the blood serum level. For the moment, that is the gold standard measure, until something something better comes along
[1] https://www.mayoclinic.com/health/vitamin-d-toxicity/AN02008
As far as really difficult, in Canada, ingesting enough pills to get to 50,000 IUs per day would be a chore. IIRC the last I checked the biggest OTC doses were 1,000 IUs. This might be different where you are.
One could see a scenario where this ends up getting consumed daily by mistake, and nutritionists would consider 50 000IU way to much in this case. (unless you have a specific condition, or post-op etc ad have been medicated in terms of therapy).