if you’re using IPv6 (nobody does) that increases another 20 bytes.
You’d typically have a receive buffer in the 1ms range though , although in theory you’d get away with 500us or even 375us on paper.
I believe (not really my area, in video we deal with timings starting in the tens in the tens of microseconds, and quantisation in the milliseconds) the rule of thumb is the total system latency between microphone and speaker have to be all within 5ms to avoid problems
Using FPGAs you can get to very low latency.
Also at some point the latency is a non-issue.
Speed of light even in fiber optics is significantly faster than the speed put sound.
And compared to TDM which is the comparison here the only thing added in packet networks latency wise is buffers, for network switches that can be nearly instant so it is only on the send and receive side.
So pretty much if you want to drop dark fiber between endpoints you can get the exact same benefits of TDM on packet switched networks without the disadvantages.
The people asking for TDM are ignoring the fact the literally several industries and some that care greatly about audio quality and latency have prettier much determined that packet switched networks are fine.
Glad someone else appreciates circuit switched technologies though :) . It really does shine in any application that isn't burst-y data transfers.