Sometimes they hallucinate them, or if they exist, sources include blatant nonsense (like state owned propaganda, such as RT) / don't support the claims made by the output.

wtf are you conversing with LLMs that you regularly are running into "state owned propaganda" in the references? my "blatant nonsense" detector is going off...

My favorite is when it cites 5 sources, and 1 of them is a real source, and then the other 4 are short form junk that point to the first one as the source. So basically its just picked one article and summarized it for you and not picked any info from any other places. Oh and bonus points when I type the exact same prompt into a search engine, and that 1 source is the top search result anyways.

what's worse is when they cite clearly LLM generated articles from web

Unfortunately, the citations are generally quite low quality and have in my experience a high rate of not actually supporting the text they're attached to.

This is on par with humans, honestly. I’ve dug into cited studies by consulting firms that were 100% false.

In my experience they just add random links at the bottom that are often unrelated to the response they give; there’s absolutely no guarantee that they did read them or that their response is based on them.