it's also not a great term for the same behavior in humans. There's the often quoted distinction by Harry Frankfurt about lying, which is knowing the truth but intentionally subverting or hiding it, and bullshitting, which is talking without any regard to truth or falsehood whatsoever.

It's a distinction worth making as people are much better at spotting lies than they are at spotting bullshit. The bullshitter may even be accidentally correct. AI models in particular will make up things in random places where nobody is sceptical or has their guard up because they see no reason for a lie, but bullshitters don't need one.

https://en.wikipedia.org/wiki/On_Bullshit

Lying is intentional. A statement which is untrue is not by itself a lie, the thing which makes it a lie is the intention of the person.

AIs do not have intentions, the only thing in an AI is a statistical model of how likely words appear in certain contexts.

"Hallucinating" is a much better term, even though it still pretends a staristical model is akin to a human.

I guess that’s true. LLMs don’t really have intentions but there are some AIs that take actions to achieve a goal that can probably be described as having intentions