Yeah an LLM is basically doing what you would do with the prompt “I’m going to ask you a question, give your best off the cuff response, pulling details entirely from memory without double-checking anything.”

Then when it gets something wrong we jump on it and say it was hallucinating. As if we wouldn’t make the same mistakes.