Sure, but do you not see the parallel with copyright here? You are assuming a priori that the baker has a right to dispose of his goods as he sees fit, and that taking that right from him is a bad thing even if the disposal would have made him no profit anyway. I see no ethical problem with such a position, but you can apply much the same argument to intellectual property as to the physical kind.
The entire concept of property is an artificial convention, which most societies have decided is beneficial and therefore worthy of respect/legal protection. But physical property is no more "natural" an idea than intellectual property. Both are just social agreements that we choose to value.
As a relevant aside, many societies/cultures throughout history have chosen not to recognise the concept of personal property and instead to hold that everything belongs to the group. In most cases, this has not worked very well on a large scale, which is perhaps why most modern societies have collectively taken the other view.
Here in the UK we have a chain of sandwich stores called Pret, at the end of the day they give their food away to the homeless - so by stealing something even if it was due to be given away, you're harming someone.