Canvas draws raster images, anything resembling an object in your drawing logic is already tracked separately by necessity. So regardless, you’d presumably check against whatever data model you’re using to determine what to draw.

By what time the user clicks, there's no reason for the program to need to remember what they drew where.

If that’s the case, what other object with coordinates would you reference on a canvas to determine whether it was clicked?