Re:Turn your picture into a Lego character
by minimaxir on Thursday November 30, 2023 @12:34PM
(#38476306)
Some notes on how you want to do this yourself:
1. Get atleast ten 1024x1024 images of a style you want (e.g. Lego). You can use SDXL/Midjourney to generate these if you are good at curation.
2. Train a LoRA on SDXL using those images.
3. Construct an image-to-image pipeline with the LoRA'd SDXL and pass along the input image as the base image. You can run for fewer steps (20-30 instead of 50) and/or preprocess the input image (e.g. segmentation).
There is a lot of other parameter tweaking for both training and generation which is where the added value is for websites such as these. But with a service like Replicate the pipeline is relatively cheap (~$0.50 for training, $0.01 for each image generation). And will only get much cheaper once people play around with LoRAs for smaller and faster models like ssd-1m-lcm.
1. Get atleast ten 1024x1024 images of a style you want (e.g. Lego). You can use SDXL/Midjourney to generate these if you are good at curation.
2. Train a LoRA on SDXL using those images.
3. Construct an image-to-image pipeline with the LoRA'd SDXL and pass along the input image as the base image. You can run for fewer steps (20-30 instead of 50) and/or preprocess the input image (e.g. segmentation).
There is a lot of other parameter tweaking for both training and generation which is where the added value is for websites such as these. But with a service like Replicate the pipeline is relatively cheap (~$0.50 for training, $0.01 for each image generation). And will only get much cheaper once people play around with LoRAs for smaller and faster models like ssd-1m-lcm.