If I'm feeling ill, should I trust Google when it says that I am probably having "internet connectivity problems", or should I go to a proper doctor instead?

"LLM says X" is the new "I Googled it and here's a copy/paste of the top result". It adds literally nothing of value to the discussion.

Unless of course the discussion is specifically about the quality of LLMs - in which case you should be vetting the answer yourself so you can actually say something meaningful about it.