I imagined this as well, and it would be pretty cool (although I actually enjoy grocery shopping).
As you say, it's still a long way off. In addition to Internet connectivity and an API, fruit/vegetables/meat and all other refrigerated items would need some kind of tracking label (RFID perhaps) so that the refrigerator could automatically track levels. Scheduling delivery of refrigerated food might also be an issue (I'm not sure how existing online grocery delivery services handle this), as I don't want my dairy items sitting on my hot doorstep all day waiting for me to get home from work.
This could be extended to pantry food items as well.
Problem is the biodegradable RFID tags don't exist. A dutch company was researching them a few years ago but that ship sailed when the iPhone 5 came out without NFC.
Google's project tango could be used with a standard library of objects, but that's still too early to tell if it's possible. Not to mention you'd be tied to Google's services and their whims.
Instead of using RFID, it's using barcodes. It's a little more manual, but same basic concept.
I'm personally waiting for my refrigerator to detect my food inventory, and auto-order grocery delivery once a week to restock itself. But seeing how Safeway, Shopping Express, Amazon Fresh, etc. don't even have an API yet, it's going to be a long wait.