We don't. You're taking the learning out of the process you're discussing by using an infinite sample. Of course, then, the machine learning part becomes trivial, and it's just an optimization problem for minimal training error!
If we have all the data, like infinite data, then we have the classification for every one of the (finite) number of possible images already. At that point, why stop at kNN? Use a hashmap!
> we have infinite sampling
We don't. You're taking the learning out of the process you're discussing by using an infinite sample. Of course, then, the machine learning part becomes trivial, and it's just an optimization problem for minimal training error!
If we have all the data, like infinite data, then we have the classification for every one of the (finite) number of possible images already. At that point, why stop at kNN? Use a hashmap!