Yes the quoted effectiveness is based on the average in practice. Anecdotally what I've seen is the failure of condoms is usually got drunk and didn't use one. In particular substance abuse is highly correlated with unsafe sex and HIV infection. I'm pretty sure it's also correlated with birth control failure.