News for hackers. Stuff that matters.
Re:Two things LLM coding agents are still bad at by kypro on Thursday October 09, 2025 @04:55PM (#45532972)
But no one uses LLMs like this. This is the type of simple fact you could just Google and check yourself.

LLMs are useful for providing answers to more complex questions where some reasoning or integration of information is needed.

In these cases I mostly agree with the parent commenter. LLMs often come up with plausibly correct answers, then when you ask to cite sources they seem to just provide articles vaguely related to what they said. If you're lucky it might directly address what the LLM claimed.

I assume this is because what LLMs say is largely just made up, then when you ask for sources it has to retroactively try to find sources to justify what it said, and it often fails and just links something which could plausibly be a source to back up it's plausibly true claims.

Re:Two things LLM coding agents are still bad at by schmichael on Thursday October 09, 2025 @05:22PM (#45533236)
I do, and so does Google. When I googled "When was John Howard elected?" the correct answer came back faster in the AI Overview than I could find the answer in the results. The source the AI Overview links even provides confirmation of the correct answer.
Re:Two things LLM coding agents are still bad at by paulhebert on Thursday October 09, 2025 @05:32PM (#45533327)
Yeah but before AI overviews Google would have shown the first search result with a text snippet directly quoted from the page with the answer highlighted.

Thats just as fast (or faster) than the AI overview

Re:Two things LLM coding agents are still bad at by schmichael on Thursday October 09, 2025 @05:48PM (#45533459)
The snippet included in the search result does not include or highlight the relevant fact. I feel like you’re not willing to take simple actions to confirm your assertions.
Re:Two things LLM coding agents are still bad at by paulhebert on Friday October 10, 2025 @12:12AM (#45535314)
When I searched, the top result was Wikipedia with the following excerpt: “At the 1974 federal election, Howard was elected as a member of parliament (MP) for the division of Bennelong. He was promoted to cabinet in 1977, and…”

To me this seemed like the relevant detail in the first excerpt.

But after more thought I realize you were probably expecting the date of his election to prime minister which is fair! That’s probably what searchers would be looking for.