This is the compositionality problem--the language model sometimes doesn't quite know how to put the words together. Better language models will help in the future; in the mean time you can give it a helping hand by prompt engineering or using img2img.