Rarely do I ever see a library that takes more than milliseconds to import, or something slowed down on a function only called on a click or every few minutes or otherwise not in an inner loop.
I see lots of slow file open dialogs, which I'm told is often because of checking for network drives and things like that. I've seen exactly one library that was causing me so much performance trouble I had to ditch it, and that was an rrules engine.
Finding the next occurrence of something like "Every five minutes on Thursday" was taking an insane amount of time, but that's because I was calling it constantly to schedule lots of different things that happened very frequently.
I've also seen lots of things be slow because they wait for the network too. But basic stuff that libraries are normally used for generally seem to be fine. I don't think there's any way I could write a faster gaussian blur than Pillow or a faster text search than the default regex libs.
The worst offenders are the "webpages packaged as programs" (electron, CEF) and UWP.
I don't think the lack of caching is the problem - the real problem is doing so much work to open the application which requires caching.
I agree with the rest but this is how we are at a point where we have to wait for windows to open. Unlike on Windows NT. It's not a problem of one thing being slow - if everything is being slightly slow, it's a death by thousand cuts.