News for hackers. Stuff that matters.
Re:Can you trust ChatGPT’s package recommendations? by ziml77 on Sunday June 11, 2023 @06:30AM (#36280003)
WTF is a recommendation in the context of a language model? It does not have preferences, just weights influenced by proximity and frequency of tokens.
Re:Can you trust ChatGPT’s package recommendations? by datenyan on Sunday June 11, 2023 @06:35AM (#36280031)
To be fair, it's realistically not too far off how most humans make "reccomendations" for packages.

Still doesn't mean that it should be trusted, though.

Re:Can you trust ChatGPT’s package recommendations? by veidr on Sunday June 11, 2023 @08:32AM (#36280776)
That is indeed the funniest (and slightly horrifying) thing. Even if you understand "glitch tokens"[1] (which is BTW the best way I've seen to explain to nontechnicals (hi, Dad!!) and also some subset of technicals (hi various friends of mine!! +_+...) that LLM applied statistics models aren't anything like sentient AGI)...

...regardless, ChatGPT and its ilk nevertheless do often already outperform "random human being coworker or uncle" at recommending just about anything... which is like Ted Chiang said[2], “That’s more a statement about how much bullshit we are required to generate and deal with in our daily lives.”

Although he was not referring to inputs from your (dumbass) coworkers/uncles, it still holds true.

[1]: https://youtu.be/WO2X3oZEJOA

[2]: https://archive.is/2023.06.04-050658/https://www.ft.com/cont...

Re:Can you trust ChatGPT’s package recommendations? by veidr on Sunday June 11, 2023 @06:54AM (#36280133)
You are of course right, but ChatGPT (and most others I presume) do recommend things in conversational English. For instance I asked ChatGPT this just now:

---

This guy on Hacker News just said, "WTF is a recommendation in the context of a language model? It does not have preferences, just weights influenced by proximity and frequency of tokens."

I mean, that is correct you don't have preferences. But you do recommend things based on those models, in conversational English.

Do you have any suggestions for how I can convince him that even though is is correct about how language models work, the utility of these models is largely derived from how they recommend solutions in conversational English?

---

And ChatGPT gave me some recommendations[1]:

---

When discussing language models and their recommendations, it's important to clarify the terminology and concepts involved. While it is true that language models don't have preferences ... they can still provide recommendations based on their training data and patterns they have learned. These recommendations are derived from the statistical analysis of...

To convince the person on Hacker News, you can explain the following points:

Language models learn from vast amounts of text data: Language models are trained on massive blah blah blah plausible-sounding sentences...

Proximity and frequency influence recommendations: Language models assign weights e blah blah blah plausible-sounding sentences... weights influence the model's recommendations by making it more likely to generate familiar or commonly used phrases blah blah blah plausible-sounding sentences...

---

[1]: https://chat.openai.com/share/a7d9e2ca-0104-469e-a6e4-6b08a5...

Re:Can you trust ChatGPT’s package recommendations? by rad_gruchalski on Sunday June 11, 2023 @09:14AM (#36281076)
Recommendations are what I use ChatGPT for. “You are X doing Y, you know A, B, C. What 20 other things should you know. Recommend reading material.” Of course those prompts are usually bit longer.