My second point is that the training of a model by definition maximizes the fitness between the final output function and the training metrics. So, if the model is trained with the watermark applied, the training process will minimize the function `model_error(input) = |watermarked_sampling(model_output(input)) - expected_output(input)|`, by definition. This means that a model trained in this way will perform better when sampled using the watermaked_sampling method than if using, say, top_k sampling.
>My second point is that the training of a model by definition maximizes the fitness between the final output function and the training metrics.
Right, but the fitness in question is watermarked text fitness, not fitness for any user interests aligned metric. You're basically saying that if we train LLMs on watermarked text they'll be really good at producing text that looks watermarked, and then we'll stick an actual watermark on top of that. Screw whatever the user wanted it to be good at.
Yes, that's the goal that was used, but they are quite simplistic and crude methods, not some specifically designed function, with carefully fine tuned parameters or something. So, if a basic function like top_k can improve model utility, it's not impossible to imagine that watermarking could also happen to do so, or at least not have a significant negative effect. So whether the effect is deleterious or not is an empirical question, not something we can assume ahead of time.
> You're basically saying that if we train LLMs on watermarked text they'll be really good at producing text that looks watermarked
No, you're misunderstanding how the training works. If we train the model's output so that it minimizes the error function after the watermark is applied on it, the model will learn how to produce the best output it can given the watermark. It will produce better text that happens to be watermarked, not "more watermarked text". Same as if you train the model on minimizing `top_k_error(input) = |top_k_sampling(model_output(input)) - desired_output(input)|`, the model will learn to produce better output under top_k sampling, not learn to produce output that's "looks more top_k".
So what? By definition with this system the LLM will chose tokens it otherwise would not, purely for watermarking reasons. Yes this token may have had a decent likelihood of being chosen anyway, but it wouldn't have been chosen and now it was for reasons nothing to do with output quality.
I'm not sure what your last paragraph is trying to say. The blue/green list system changes what output the LLM would otherwise produce. You can't train it to produce watermarked output with this system. If you tried to, there would be no delta between trained output and watermarked output for you to be able to detect.