What is missing that they aren't universal? Is there some kind of tradeoff between full power load and full data transmission capacity or something?
As far as I know a universal cable is not possible. My reading of the spec says that it should be but I am not a USB implementor and some of my friends who are tell me it's not possible. In any case nobody ships such a thing.
The charging cables provided with devices are likely to be power-only. With the 87W charger, Apple ships ones that can carry about 100W (the maximum per USB spec -- 20V@5A) which requires thicker conductors; my understanding is that those cables don't carry data (i.e. they can safely be plugged into any third party brick without you worrying that the remote device will attack your computer). You want a power-only cable for this purpose. I am not sure of they have the same cable with the smaller adaptors.
The very high speed cables only work over short distances unless you go to optical (which I'm not sure anybody has deployed).
The most standard cables carry "USB 3.1" which is just another name for USB 3.0 carried over Type C connectors. The higher frequency and higher power cables are more expensive to design and build so will cost the customer more. That's one reason you have to specifically look for cables for DP, PCI-e/Thunderbolt etc.
No, it's not; USB 3.1 adds the new SuperSpeed+ mode over all supported connector types (and USB 3.2 adds two additional SuperSpeed+ modes for Type-C connectors.)
USB Type-C connector spec is a separate spec issued after USB 3.1, USB 3.1 isn't USB 3.0 over Type-C connectors.
Overall, I think a quick glance at the whole discussion under my comment proves my point quite well.
Things like charging your phone can be done with USB C.
Things like external hard drive are generally done with Thunderbolt
Things like External GPU's have to be done with a Thunderbolt cable that is less than 1m
All of that said, I am not 100% sure of all of that, because it is all so confusing.
If you ignore Thunderbolt, can you always rely on the latest USB-C cables Apple and Google sell working for everything? Or are there still issues?
This affects everyone.
[0] https://en.wikipedia.org/wiki/Thunderbolt_(interface)#Thunde...
Dating back to the early days of Apple, they have been hyper conformant with the spec (e.g. no drivers needed to connect to conforming devices). They still are.
I answered your question about which cables above.
Why? External USB drives are much more common.
TL;DR: USB drives are really really slow.
625 MB/s is 5Gb/s; the switch from bits to bytes seems to be a way to exaggerate the difference from Thunderbolt speeds quoted in GB/s; at any rate, USB 3.1 Gen 2 (equivalent to USB 3.2 Gen 2×1) 10 GB/s cables and device support isn't hard to find; I don't know that anyone had USB 3.2 hardware (controllers, or cables supporting the new 2×2, 20Gb/s, mode) yet.
Currently there are four common speeds/specs for the high-speed lanes: none, USB 3.0 at 5GHz, USB 3.1 at 10GHz, Thunderbolt 3 at 20GHz.
Cables that can support the higher speeds without signal loss issues tend to also be increasingly short. That's another issue getting in the way of an ideal cable; it's hard to carry such high frequencies to the same distances.
There's no tradeoff between speed and 100W support, but most cables do lack 100W support for whatever reason.
A lot of cables (even from Apple and Google) pick the "none" option for high speed lanes. Such a cable can only run at USB 2.0 speed and is often called a "charging-only" cable. (True charging-only cables are significantly more rare. They also probably violate the spec.)
Unfortunately that isn't enough yet, as there is no "universal" cable, and may never be. You can have an excellent cable that nonetheless fails for what you plugged it into, or (potentially worse) falls back to something that kind of works (e.g. a slow data transfer through a port that supports PCI-e).
The biggest botch the committee made was not mandating connector labeling. I'd have gone with colors, but not everyone can see them. If the cable had resistor-style stripes to indicate what it supported, and if the ports on the host devices did likewise we'd be in much better shape. Even some licensed logos would have helped though the connectors are so tiny.
Then again the names the USB committee has come up with in the past (high speed, SuperSpeed, 3.0, 3.1 etc) have uniformly been confusing rather than elucidating.
In my case I agree: I have had nothing but success with Type C, and it has simplified my life (except for the lack of high power chargers). But I had to learn more than most people would or should have to in order to get there.