Exactly! Often it is simply impossible for the compiler to know. In this respect it is similar to std::unreachable.
PGO is also a pain to use in some situations. You need to be able to regularly exercise all of the main paths in the program under instrumentation, preferably automated, using a configuration as close to release build as possible. That's hard to do when your release build lacks automation support, has nondeterministic behavior by design, cross-compiles to another platform, or requires networked services to exercise main paths. I don't even know how people deal with PGO when there is a requirement for deterministic builds.
[1] https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p04...
and look at code in the wild
[2] https://github.com/search?q=[[likely]]+language%3Ac%2B%2B&ty...
PGO is a mixed blessing and detracts a bit from the thrust of the article. The more obvious conclusion is to continue using builtin_expect (on the boolean guard of a branch) which works great and has done for ages.
However better not give data to the function that contradicts the condition if you don't want to figth nasal daemons.
Edit. It's because assume does not map to expect, it maps to builtin_assume. So that's just another way to write undefined c++.