Possibly an analogous situation is the history of steel in Japan and their efforts to extract iron from sand, since they don't have significant iron ore to mine on their island.
A city decayed to rust is going to be a thin layer of iron spread out over miles with some hot spots like where a building once stood (but presumably without a map of the city in this distant future scenario), but there won't be a vein of concentrated ore. Distributed rust can definitely be turned back into pure iron but the energy requirements are going to be substantially higher to do so since you're going to have to sift through much more material to collect it, more material to separate and concentrate it, more material to smelt off, and your operations will have to be more mobile to retrieve it over a larger area. That's why I think retrieving iron from our society will be more like extracting iron from ironsands (2-20% iron), and it will have similar effects on that subsequent society that sits between us now and some future point where geology has re-supplied it to the surface millions of years from now.
FWIW those are the ratios for the oxides themselves but the formations are not necessarily huge piles of pure oxides, if you go a bit lower to the "sources" section the lowest-concentrated formations viable for exploitation are
> Banded iron formations (BIFs) are sedimentary rocks containing more than 15% iron composed predominantly of thinly bedded iron minerals and silica (as quartz).
However that's only for post-industrial societies, at least if you have alternatives, as it requires churning through ridiculous amounts of materials.
When you don't have alternatives the ironsand article (which would be used in places with no good or accessible ore deposits e.g. japan, famously) quotes
> Sand used for mining typically had anywhere from 19% magnetite to as low as 2%.
though much like gold panning the ironsand would be sluice-separated to a concentration of 30-50% before it was further processed.
Most ironsands deposits are not considered financially exploitable to this day though, with the exception of NZ's where the iconic "black sand" beaches of north island are extremely rich in magnetite (up to 40%).
They're concentrated rust mixed with rocks, otherwise it's not economically viable to extract.
Like, iron is ridiculously common, relatively speaking: on earth as a whole it's more common than oxygen, for the crust it ranks 4th at 5% by mass, meaning if you went at it randomly you'd need to sift through 20kg of materials to get 1kg of iron.
Currently, we exploit formations as low as 15% iron (banded iron formations / taconite), that's the lower limit of the economically feasible, and those results in absolutely enormous amounts of tailings (waste materials).
Pre-industrialisation, unless you had no other choice (e.g. only had ironsands to work with) you really wanted to exploit natural (or "direct-shipping") ores, in the 60~70% range, the extraction is way too much work otherwise.
Fortunately, despite millions of tons of production every year, we are nowhere near using up the ore.
Might depend how long it takes: because rust is porous and friable, rusting iron should eventually degrade to nothing, and the rust would be difficult to re-concentrate then reduce back to iron.