I doubt that any pro-cyclist wants the speed advantages of one if it means that cornering is more difficult, veering into dangerous.
I am also pretty confident that the 'manoeuvrability' disadvantage is completely irrelevant for any individual track event. And do you think that it would be an important factor for time trials?
[edit] And yes, for time trials recumbents would be nice, but already time-trial bikes are pretty different and unsafe, I don't think the UCI would allow even more risk.
Maneuverability is important for time trials. Most courses have several tight turns and crashes are common.
Things like better shifters, lighter frames (within reason) more aero parts (within reason) are still allowed.
The UCI's 6.8kg weight limit is much higher than what is achievable with modern materials. So much so that there are frame builders who can build you an "illegal" bike made out of _steel_: https://www.rodbikes.com/catalog/outlaw/outlaw-main.html
With a minimum limit, no one feels the need to make their bikes less than perfectly strong. And when every OEM is making frames at perfect strength under the limit, they drop it.
https://en.wikipedia.org/wiki/Recumbent_bicycle