Re:fMRI-to-Image with Contrastive Learning and Diffusion Priors
by RC_ITR on Tuesday July 18, 2023 @12:29PM
(#36775157)
How is contrastive learning done with one model, exactly?
I agree only one is used in inference, but two are needed for training (otherwise how do you calculate a meaningful loss function?). Notice in the original CLIP paper, there's an image encoder and a text encoder, even though only the text encoder is used during inference. [0]
I agree only one is used in inference, but two are needed for training (otherwise how do you calculate a meaningful loss function?). Notice in the original CLIP paper, there's an image encoder and a text encoder, even though only the text encoder is used during inference. [0]
[0] https://arxiv.org/pdf/2103.00020.pdf