TCP has a lot of tunable parameters where the choices are baked into it, and it assumes that extra round trips are no big deal.
Crypto does not have all that many tradeoffs. You just need expert implementation. It's dangerous and unnecessary to do it yourself.
For something that is low-bandwidth and latency-sensitive, you can take TCP and make minor adjustments and come out with something that is far better suited. Even something as dumb and easy as sending all packets twice could turn an semi-common jitter into an ultra-rare jitter.
If you could open a TCP socket with forward error correction options, and disable head-of-line blocking, then you could probably argue against custom TCP-like protocols. But that's not the world we live in.
You don't need to know that much about networking to get something that's better than TCP in this specific use case. It may be unoptimized and terrible but your goal is latency. Just pretend that 5% of packets will get lost and do something to compensate, and you beat TCP on imperfect networks, even as a novice.
WebRTC went for UDP-encapsulated SCTP in order to get a transport without HOL blocking. I'd look into that or an existing UDP game net transport (like Enet) first.
https://packetzoom.com/blog/lessons-learned-tcp-slow-start-d...
I took this to be something more akin to "never implement your own crypto." If everyone followed that advice there would be no crypto, so it's obviously not blanket advice.