The Virginia part was already retroceded in 1847, less than 60 years after the creation of the district; retroceding the Maryland part is a pretty common Republican alternative to D.C. statehood since pressure began for it (though it is opposed by both Maryland and D.C. residents who have 232 years of separate history.)
I also don't think that proximity really confers any undo influence at this point - so if the federal buildings just became federal property within the state of VA/MD similar to thousands of other federal properties that'd probably be fine.
I'm sure many locales with 670k residents would love to become states, that'd be a pretty sweet deal.
We have a well established process for asking states about this sort of thing, the US Congress. They don't seem to be wild about the minimum populous state standard, we should ask about the median populous state standard.
Just because you bought someone a coffee after punching them in the gut doesn't mean they appreciate your patronage.