I've been curious, isn't img2img basically just more SD passes? Ie you generate something with 5 passes, then img2img just runs the other 95 over that to "complete" the image.
Basically, as far as I understand it (which is not much), txt2img generates a very rough blob from a prompt, then img2img does additional passes over that to refine it to something that has much higher quality. Is this incorrect?
For example I generated [0] as my base image, I passed it into a dithering filter to make it look more "retro" and got [1] (was going for retro PC98 visual novel style backgrounds), and then [3] are all new results generated from it (I added stuff like "village" or "trees" etc in the new prompt and also made the resolution widescreen). You can tweak parameters like the prompt, the depth (ddim), the "strength" of the original image, and the "scale" (not sure what it is but it does change things around). Then you can play with that and even iterate on more images.
[0] https://upload.i.ng/file/goaltendersgrievousSumter/seed_6436...
[1] https://upload.i.ng/file/overstatementrearwardsplighting/dit...
I'd be really interested to see this work extended with something that transmits (say) a 64x64 pixel copy of the original image and uses it as source material for Stable Diffusion to work on.
Depending on the parameters it might be a legitimately viable image compression algorithm, at least for some types of images.