In general I agree with you. But it gets trickier once you look into more automated content generation. It's been shown already that in natural language generation, as an example, big model frequently regurgitate full text passages. Similarly in computer code generation.