No resource is truly finite except for energy, so there are upper bounds for the cost of "finite" resources. Pretty much all things that we dig out of the ground don't leave the planet when we're done with them, they just get diluted. So the cost of throwing it away is upper bounded by the cost of extracting the material from a dilute source, e.g. the ocean. That's usually prohibitively expensive.