I'm guessing you're thinking about things like Swiss Tables. But the situation isn't that Swiss Tables "optimize" the STL's containers instead they're just the replacement you'd actually want. You can't "optimize" the STL containers, because they're defined in a way that's hostile to optimization.
Take std::unordered_set. Why is it so slow ? Well, your standard library is obliged to make this work roughly the same way as it would have when explained in a CS introductory Data Structures class in the 1980s. This is not necessary for the ordinary understanding or use of an "unordered set" which is why Swiss Tables has one that's much better, but if you've paid attention in that class you know there are buckets of keys with similar hash values so that's what the STL is obliged to provide.
If you just want an "unordered set" you do not want std::unordered_set despite the name, you want the much better replacements from Swiss Tables or various other offerings and it's unfortunate that std::unordered_set is in the standard and those are not.
Of course the other reason std::unordered_set is so slow for you isn't solved by Swiss Tables, but it was called out by the Googlers presenting Swiss Tables more than once. Your hash function is garbage. Even if you insist on using std::unordered_set because it was good enough thirty years ago or whatever, this part of the lesson is invaluable anyway.
When using data structures that are faster because of hashing, you defeat them by using poor quality hashes. To a first approximation if you aren't sure that you are using a good quality hash then you probably aren't. In any optimisation quest start by measuring, and in this case that means measuring: Is your hash actually any good at... hashing ?
“Folly has chosen to expose a fast C++ class without reference stability as well as a slower C++ class that allocates each entry in a separate node. The node-based version is not fully standard compliant, but it is drop-in compatible with the standard version in all the real code we’ve seen.”
Neither of the Google C++ Swiss Table presentations I've seen were about how somehow std::unordered_set can be replaced by this completely different thing. The existing container is just useless, that's sad but there's nothing to be done about it because the ABI break cost is unacceptable.
Instead the talks were about why this thing (Swiss Tables) makes sense, how you can make use of it in your own software, and maybe about tricks you can use in your own data structures, or about how Hyrum's law interacts with this work.
Not all Googler talks.
My talks at CppCon (Polymorphism != Virtual https://m.youtube.com/watch?v=PSxo85L2lC0, Beyond Struct: Metaprogramming a Struct Replacement https://m.youtube.com/watch?v=FXfrojjIo80) had nothing to do with Google3 and use techniques that would likely be discouraged by the style guide (especially the struct one).
For corporate sway in research - its my personal opinion (any only limited to me), that citizen awareness is generally high. HN and similar communities are quick to spot gaping holes or flaws, and alternatives are plenty as well. There is fortunately a still healthy ecosystem of indie developers who contribute everything from Linux kernel to iOS patches. As you mention, as of present this does not seem a big concern and the alternative scenario (no academia-industry symbiosis) could be worse.
The question of how much we actually avoid this is a complicated one to answer. I like to think that a lot of the obvious biases get caught, but I have also been around long enough to know that Hacker News is definitely not immune to this. My employer constantly falls into the trap of having a problem and then looking around to see how FAANG is solving it, then trying that solution largely uncritically, despite not quite being a FAANG. It's mildly amusing when your see a principal engineer with several times more experience (and compensation!) than you do get tripped up by it, but it only emphasizes that evaluating research with a critical eye is difficult to do and everyone struggles with it to some extent.
Slightly off topic :) I feel by the time people become Principals, they lose the laser sharp focus because they are juggling too many things at the same time. Principals who work as IC on the team, however are much better since they are hands-on to the current problems.
For example, if you ever attend a C++ talk by Googlers, you’ll notice that they basically only talk about C++ as they use it, silently ignoring things they don’t care about. By virtue of google3 and their style guide, things like ABI compatibility are of very little consequence to them, and they can take away expensive-to-support APIs and present about how they “optimized” some part of the STL, or how support for exceptions is something they’ll look into later or (when they’re feeling uncharitable) bad actually™. Similarly most talks about Linux networking are driven by e.g. Facebook, who seem to slowly just be converging on running their entire stack using eBPF in the kernel. Apple presents their ML research about on-device learning and somewhere in the middle they’ll be like “oh also we have specialized silicon to do this efficiently otherwise it isn’t practical”. Microsoft will present virtualization research and you’ll find that their threat model is trying to prevent people from jailbreaking the Xbox.
I’m not trying to say this research is bad or not useful, but it’s important to put them in the context of where they’re coming from or who they’re being funded by, because the entire thing-from the premise, to the execution, all the way to the conclusion-is going to be dependent on the circumstances the research was done in, and often it’ll be presented as a general result when it really only make sense in the context of that particular company’s needs. If you’re being cynical, it’s a way to appear open and exercise soft power through mindshare, but for most cases I think the alternative (no research) is probably worse so I’m not generally concerned.