Re:Semantic ablation: Why AI writing is generic and boring
by writeslowly on Tuesday February 17, 2026 @03:18PM
(#47052717)
I wonder if you can use lower quality models (or some other non-llm related process) to inject more "noise" into the text in between stages. Of course it wouldn't help retain uniqueness from the original source text, just add more in between.
Re:Semantic ablation: Why AI writing is generic and boring
by causal on Tuesday February 17, 2026 @03:00PM
(#47052430)
I’m not convinced removing RLHF would really make the probabilities generator give us distributions that can diverge from the mean while remaining useful.
In other words, this might not a problem that can be overcome in LLMs alone.
Re:Semantic ablation: Why AI writing is generic and boring
by anon-3988 on Wednesday February 18, 2026 @04:26AM
(#47059017)
Be honest, did you use LLM to write this reply? Am just genuinely curious as I am trying to gauge and tune my senses.
OT: I do agree with your assessment.
Re:Semantic ablation: Why AI writing is generic and boring
by jannyfer on Wednesday February 18, 2026 @11:56AM
(#47063144)
GP does use a lot of "it's not this, it's that" pairs. :)
Re:Semantic ablation: Why AI writing is generic and boring
by mizzao on Wednesday February 18, 2026 @12:37PM
(#47063705)
The only AI flavor was "Semantic ablation isn't a side effect of the training process, it's the intended outcome of the objective."
But, there are hyphens used where em-dashes are expected, so that is anticorrelated with LLM writing.
The more interesting cause I think: RLHF is the primary driver, not just the architecture. Fine-tuning is trained on human preference ratings where "clear," "safe," and "inoffensive" consistently win pairwise comparisons. That creates a training signal that literally penalizes distinctiveness - a model that says something surprising loses to one that says something expected. Successful RLHF concentrates probability mass toward the median preferred output, basically by definition.
Base models - before fine-tuning - are genuinely weirder. More likely to use unusual phrasing, make unexpected associative leaps, break register mid-paragraph. Semantic ablation isn't a side effect of the training process, it's the intended outcome of the objective.
Which makes the fix hard: you can't really prompt your way out of it once a model is heavily tuned. Temperature helps a little but the distribution is already skewed. Where we've gotten better results is routing "preserve the voice" tasks to less-tuned models, and saving the heavily RLHF'd models for structured extraction and classification where blandness is actually what you want.