A quick google seems to show that the cost of milk has increased over time.
http://www2.census.gov/prod2/statcomp/documents/CT1970p1-06.... (page 31)
Data from 1890 - 1970.
That's not why.
When costs go up, producers become less willing to supply product at a given price. This shifts the supply curve left, and establishes a new equilibrium at a lower quantity but higher price. Of course, with regards to oil, demand has been exploding in developing nations, which has pushed the demand curve to the right. Now, picture pulling the supply and demand curves to the left and right, respectively, at the same time to see how that causes the equilibrium price to spike.
As for speculation, it's hard to tell what speculation is really doing here. Speculation is, to an extent, the mechanism through which the market incorporates knowledge about the future scarcity of a good into the price of the good. Whether or not we've hit "peak oil" we're clearly in a situation where oil extraction is getting more complex and expensive and demand is exploding. And oil resources are clearly finite and we'll run out sooner or later and there is no alternative in sight. Given these factors, the price of oil should be exploding.
But the insane volatility of the market, and rising prices completely out of proportion with the slowly, steadily increasing cost of extraction, is most definitely because of speculation.
And come on, how is that even slightly controversial? When someone with the capital to affect the price is in a position to profit whether the price goes up or down, what do you expect to happen to the price?