News for hackers. Stuff that matters.
Re:Language Models Are Super Mario: Absorbing Abilities from... by og_kalu on Saturday April 06, 2024 @02:19PM (#39954311)
Let's say i trained a model on the artworks of fiona staples that is invoked by typing "fiona staples style". Then i trained a model on the artworks of james daly that is invoked by "james daly style".

What i want when i merge such models is a model that can generate in the artsyle of either fiona and james independently or a mix of both if i specify both keywords in the same prompt.

Currently, if you merge these 2 models and generate "busy city street, fiona staples style", you will not get a model that can generate works in fiona's style, you will just get a model that will generate an odd mix of fiona and daly even if you only specify one of them.

It means you either need to train a million different models for a million different concepts with no chance of cross usage (e.g x person wearing y clothes in z style will not be possible) or train those concepts at the same time, which becomes very cumbersome requiring a retrain of n+1 concepts on a fresh model anytime you want to introduce a new concept.

Oh, and training on x then training on y doesn't work in practice either because the model will mostly forget x learning y.

There are no comments on this comment yet.