Author here. Yeah you're basically right. GANs vastly improve the situation though because being safe with "green for grass, blue for sky, brown as default" doesn't work in the Generative-Adversarial setting. The critic will assign lower scores if the generator keeps outputting brown. Now I'd think the generator would get more creative than going for blue constantly but that might just be a matter or more/better training (...?)