LLMs can make social consequences for us, from creating and maintaining state about opinions of us individually, and we will want to coexist in that reality. That's enough to adjust to and... respect. It doesn't seem to matter what arbitrary threshold of "intelligence" there is, in the face of an entity that creates much less arbitrary and dynamically generated - but predictable - outcomes. I don't have to understand what a human was trained on, or how smart they are, to understand the same thing. The same with an animal.
On HN intelligence is often seemed to be defined as more about new and novel thought / invention - it’s only intelligence if it thinks a thought that has never been thought/written before.
Off HN ‘intelligence’ seems to be closer defined to “is it smart enough to replace my job as a knowledge worker”. Pharmacists would say it has intelligence if it could automatically check prescriptions, and data analysts would say it has intelligence if it can answer a data request without their intervention.
IMO most people seem to define intelligence as whatever the part of their job is that currently requires analytical skill, because people are worried about the impact this could have on the middle and entry tier of knowledge work, rather than defining AI as needing to invent something fundamentally new.
These goalposts have literally been sent to space.
Not especially long ago humanity knew basically nothing. The epitome of technology was the stone age. Literally crush two rocks together and then poke things with the sharp bits that flake off. Somehow, in the blink of an eye, we went from that to putting a man on the moon.
So let's do the same with these sort of LLM models and see what happens. Train one with the entirety of expressible human knowledge from the stone age. Where will it send you? I think the answer is largely pretty much where you are. In any case, certainly not to the moon. It seems we're just building natural language search.
This isn't shifting the goal posts. The problem is we keep intentionally making meaningless goal posts in pursuit of AI, because if you don't then the goal seems impossibly far off. But that's because the reality is that it probably is!