Yes, the network has picked out some image features. But those image features are being trained towards outputing what are random predictions, so the features are essentially noise.