There are massive areas off US coasts with proven reserves that are legally untouched (CA moratorium after Santa Barbara spill only allows for marginal/incremental pumping). In the time between those reserves being shut off and now, the economics of retrieving that oil changed completely, and there is now a ton of "easy oil" off the california coast.
Nobody's arguing that oil is not an infinite resource, but I can point to a single location that contains immense amounts of easily tappable oil.
While the original motivation for the CA moratorium was ecological, these days it's (implicitly) economic. All CA has to do is wait until the price spikes start happening, and dribble out oil at very high prices while hoping there isn't another Deepwater Horizon.
The proven reserves for California are based on historic estimates, they haven't been thoroughly re-evaluated, and were based on previous extraction technologies (before side-drilling etc). The current position of my friends who do oil/gas analysis is that California proven reserve estimates are at least 1 order of magnitude below reality mainly because people aren't actively looking because of the moratorium.
The point of all this is that California's offshore fields aren't all that special, and they aren't "easy oil". If they were, any environmentalist resistance would have been defeated long ago.
I know you said this in reference to Californian oil, but it also applies to Saudi oil. The number of dollars paid isn’t the only cost of using that oil.
At least install an effective blow-out protector.
Now, I probably should have picked a better comparison, since oil really is zero-sum in ways that needing to grow more food isn't.
The point is that in an alternate universe where agriculture was 40% less efficient it doesn't follow that there would be 40% fewer people. Rather, more of the gross world product would be spent on agriculture, since there's a lot you can do to throw more money at the problem to increase yields, and pricing would do a lot to increase the efficiency.
I disagree with the idea that there'd be proportionally less meat, for a couple of reasons. First, a lot of grazing-animal meat (beef, basically) comes from land that is unviable for plowing or other vegetation farming. Second, in the absence of inexpensive chemical fertilizer, manure would be a much more important fixer, just like it has been for thousands of years.
Peak oil is a matter of when, not if. It is not an infinite resource. The interesting (and worrisome) question to me is if we'll be able to convert the transportation grid to predominantly electric (and thus powerable by solar/wind) before we hit a global peak and start losing maximum recoverable oil at 3-10%/year, which will cause intense price spikes and resource wars, due to inelastic demand.