No, the fact that others have learned it is not (strong) evidence that you can learn it too; you would need to know how similar you are to those people, and whether those dimensions are relevant.
IOW, exactly the diagnostic criteria I suggest we search for rather than immediately skip to "so what, other people thought that too".
>Your self-doubt about it is based on (falsely) thinking that you are individually incapable of doing a thing plenty of others have done.
That's generally not what is happening in practice. People don't de novo say "I can't ever do this". Rather, they attempt (or otherwise survey) it, find difficulties ("I can't consistently understand how the stack is doing that"), see others that have no similar difficulties ("what? It would have taken me a day to work through that and it comes naturally to you"), and on that basis conclude that they're unlikely to succeed as well.
"You have Impostor Syndrome" doesn't make any headway on the core problem.
That is, if you believe that you can't learn kernel hacking, that's (potentially) impostor syndrome. Lots of people have in fact learned kernel hacking, you can learn it too. Your self-doubt about it is based on (falsely) thinking that you are individually incapable of doing a thing plenty of others have done. But if you believe you can't build a perpetual motion machine, that's just thermodynamics. Nobody has built a perpetual motion machine, and doubt about your ability to do so (which isn't really self-doubt) will be quickly confirmed by a survey of the literature.