img2img and in/outpainting are already two forms that this idea exists in. Maybe not with textual labels in the image itself, but the concept of having a starting image that a model can modify for a desired change is definitely already a thing.