News for hackers. Stuff that matters.

A comment on Modern-Day Oracles or Bullshit Machines. Parent comment

Re:Modern-Day Oracles or Bullshit Machines by rcxdude on Sunday February 09, 2025 @09:03AM (#42990751)
Where in the loss function of LLM training is the relationship between their model of reality and their predicted tokens? Any internal model an LLM has is an emergent property of their underlying training.

(And, given the way instruct/chat models are finetuned, I would say convincing/persuasive is very much the direction they are biased)

[ Parent | on HN ]
Re:Modern-Day Oracles or Bullshit Machines by TeMPOraL on Sunday February 09, 2025 @02:05PM (#42992777)
> Where in the loss function of LLM training is the relationship between their model of reality and their predicted tokens?

In the part where their loss function is to predict text that humans would consider a sensible completion, in a fully general sense of that goal.

"Makes sense to a human" is strongly correlated to reality as observed and understood by humans.

[ Parent | on HN ]