The Elder Scrolls games through Oblivion (and I think Skyrim?) are not permanently persistent. The game does eventually do garbage collection, and only some places in the world are safe from it. This is why every major town/city has a house you can buy; your collection of every cheese wheel in the game may not persist forever if you leave it in some random chest in a shop.