No way for Swift to "let go". It solves a problem (having a modern language for OS X to go forward") that Objective-C doesn't.
Besides swift makes sense even without a new set of frameworks. People use it already and are productive. Sure, Swift inspired frameworks would be a better fit, but that not a show-stopper.
GC wasn't that much a deal to let go, because they found a way to do the same thing without it (ARC). It's not like they had to rewrite anything for the new post-GC era.
The official recommendation from Apple was to not transition for GC but only to use it for new projects or for complete rewrites. So it was a big deal for every developer who relied on GC.
And only because something is new does not mean it is modern. All of the concepts that Swift implements are fairly old. Swift is just new.
Though, look at QuickTime for example. That was deprecated in Mavericks, but it's going to be around for another few releases. AVFoundation (the framework I use the most) is used by FinalCut, iMovie, etc. so porting that isn't straight forward because it would break so much stuff. So I do think you'll see deprecation of certain things as maybe they move to design patterns that are a better fit for Swift, but entirely replacing AVFoundation would be straight up insane when it works so well (as long as you are doing everything by the rules laid out in a largely unwritten book).
So I think QTKit is an edge case and probably not used by a ton of developers directly. I think the 80/20 rules also applies here... by porting 20% of the frameworks you get 80% of the functionality.
That being said: I think that if it happens it will happen over time. Deprecating something in Mavericks does not mean that it has to be gone in the release after Mavericks. GC was with us for way too long... But you can replace almost anything within lets say 5-7 years. That would be 5-7 major releases.
If your app used GC then everything your app links against or loads at runtime has to be GC-compatible. So Apple had to make all of it's frameworks GC-compatible. But many 3rd-party frameworks didn't.
Also: If your app used WebView to load a web site which contained something with flash (YouTube, ...) then your GC app would crash immediately because WebView loaded the Flash plugin which was not GC-compatible.
Thus using a WebView to load arbitrary sites was incompatible with GC.
And Apple is certainly in a better position to do such ports than you were back then: They have the full source code of every framework they developed. I am not saying it is trivial or not much work but it is doable over a couple of years... don't you think? You have certainly more internal knowledge than I do.
But I would like to mention that people seem to overestimate the work required to "port" a framework from one language to another. Take 280North as an example. A tiny company (funded by ycombinator) founded by 2 ex Apple employees. They have developed a JavaScript variant of Objective-C (called Objective-J) and reimplemented the most important parts of AppKit. It took them a couple of months but in the end it all worked out, many people used (and are still using it) Cappuccino (name of the framework) and 280North was bought by Motorola for something like $20.000.000. So two guys => 1 language, a couple of frameworks in a matter of months.
Also I am sure you are aware that every year Apple introduces a ton of new frameworks and deprecates some frameworks. It would only take a couple of releases to deprecate 80% of the frameworks used by most apps and introduce replacements for them.
Swift without a new set of frameworks makes little sense IMHO. So either Swift will dies - just like GC did or it won't and then we will see new frameworks written in Swift.
Talking about GC… When Apple introduced GC they told us that it was the latest and greatest shit. It was the future. When introducing GC they told us that "hey - we have ported Xcode to GC - look it is that easy.". So they had more apps using GC from day 1 than using Swift from day 1 (well the WWDC app written in Swift does not count…). Apple had to ship GC-capable versions of their frameworks from day 1. They were even more committed to GC than they are now to Swift so it would not surprise me at all to see Swift let go in a couple of years and buried deep.
At the moment everything seems possible.