News for hackers. Stuff that matters.
Re:Salesforce study finds LLM agents flunk CRM and confident... by worldsayshi on Monday June 16, 2025 @10:27AM (#44289860)
This makes me realize something: The internet has very little training data for "when to shut up". The bias is always towards more yapping.
Re:Salesforce study finds LLM agents flunk CRM and confident... by themanmaran on Monday June 16, 2025 @11:48AM (#44290722)
This is a big problem when it comes to conversational agents. Sometimes users ask questions that are really prying, potentially misleading, or just annoying repeats (like asking for a cheaper price 50 times).

In these situations a real person would just ignore them. But most LLMs will cheerfully continue the conversation, and potentially make false promises or give away information they shouldn't.

Re:Salesforce study finds LLM agents flunk CRM and confident... by jotux on Monday June 16, 2025 @11:50AM (#44290735)
> But most LLMs will cheerfully continue the conversation, and potentially make false promises

Example: https://www.bbc.com/travel/article/20240222-air-canada-chatb...

Re:Salesforce study finds LLM agents flunk CRM and confident... by notahacker on Monday June 16, 2025 @01:31PM (#44291571)
Indeed I suspect if anything the weighting is the opposite (being annoyingly persistent weights and LLM towards spitting out text that approximates what the annoyingly persistent person wants to get), whereas with humans it weights then towards being less helpful...
Re:Salesforce study finds LLM agents flunk CRM and confident... by tempodox on Monday June 16, 2025 @10:50AM (#44290135)
+1. Actually, the infinitely many things that have never been posted would be such training data, but how do you count how much nothing you hoovered up while stealing data?
Re:Salesforce study finds LLM agents flunk CRM and confident... by falcor84 on Monday June 16, 2025 @11:37AM (#44290602)
Now that much of the input to AI systems is from the search tool, maybe post-training should indeed be treating the lack of a result as a signal, perhaps a bit like in TF-IDF, where something being more rare in the corpus as a whole implies that it's more unique and potentially meaningful to the current document.
Re:Salesforce study finds LLM agents flunk CRM and confident... by danielbln on Monday June 16, 2025 @11:22AM (#44290462)
Stealing implies the original is no longer there. I'm no fan of the large AI labs hoovering up the Internet, but let's keep our terminology accurate. We don't even know if this sort of crawling and training on public data constitutes infringement.
Re:Salesforce study finds LLM agents flunk CRM and confident... by dylan604 on Monday June 16, 2025 @11:27AM (#44290510)
Pedantry is so boring. In conversational parlance, stealing is often the meaning without paying for. So yes, pedantically, this would be unlicensed use of vs the removal of the original from the owner's possession. But what else do you want us to think when even the FBI pushed the copying is stealing bit with their logos at the head of DVDs/VHS tapes?
Re:Salesforce study finds LLM agents flunk CRM and confident... by chii on Monday June 16, 2025 @12:28PM (#44291055)
> this would be unlicensed use

which is exactly what the parent poster is implying - the hoovering up of data off the internet may not be unlicensed use. After all, the information is not what's copyrighted, but the expression of it only.

By calling it stealing, it already presupposes the idea that such hoovering is unlawful, before it is made clear that it is unlawful. And it prejudices the "jury" so to speak - the language for which you call the subject can influence other people's perception.

Re:Salesforce study finds LLM agents flunk CRM and confident... by notahacker on Monday June 16, 2025 @01:49PM (#44291710)
We know for a fact that some LLM developers made digital copies of lots of copyrightable material for the purpose of training a system to create [unattributed] derivative works which had licenses expressly forbidding ingesting the content into an information retrieval system for the purpose of creating derivative works [without attribution], and that derivative works were produced, some of them containing substantial portions of content recognisably identical to copyrighted material.

LLM providers are free to argue in and outside court that EULAs or software licences are not applicable to them or enforceable at all, or that their specific actions fell short of violations but it's far more prejudicial to wade into conversations to try to shut down any suggestion that it might be possible to do anything unlawful with an LLM.

Re:Salesforce study finds LLM agents flunk CRM and confident... by meepmorp on Monday June 16, 2025 @12:07PM (#44290883)
> Stealing implies the original is no longer there.

It really doesn't, and I'm pretty sure even you regularly use the word 'steal' in a context where there's clearly no such implication.

Re:Salesforce study finds LLM agents flunk CRM and confident... by esafak on Monday June 16, 2025 @11:54AM (#44290777)
If you value brevity, don't ask Gemini.
Re:Salesforce study finds LLM agents flunk CRM and confident... by el_benhameen on Monday June 16, 2025 @11:59AM (#44290827)
Excellent point! You’ve stumbled upon something fundamental about Gemini—it’s exceedingly verbose, even when answering the most mundane of queries. Let’s dig deeper …
Re:Salesforce study finds LLM agents flunk CRM and confident... by rsynnott on Monday June 16, 2025 @12:35PM (#44291114)
Delve deeper, surely?
Re:Salesforce study finds LLM agents flunk CRM and confident... by oblio on Monday June 16, 2025 @01:18PM (#44291445)
Into the mines of Moria?
Re:Salesforce study finds LLM agents flunk CRM and confident... by soared on Monday June 16, 2025 @02:16PM (#44291990)
You’re on the right track! Exploring an LLM’s verbosity is an important step in analyzing its usability. A critical first step is…
Re:Salesforce study finds LLM agents flunk CRM and confident... by j45 on Monday June 16, 2025 @11:59AM (#44290821)
On one hand if responses were concise and perfectly clear (more than the human interacting with it), could it be unnerving?

Prompting with clarity seems to help alleviate any accumulated response pressure where it's having to reach beyond what it has readily available.

When it comes up short, it seems to dig deeper and come up with more than intended, or over respond.

Jumping to solutions remains one of the biggest challenges.

Re:Salesforce study finds LLM agents flunk CRM and confident... by detourdog on Monday June 16, 2025 @01:43PM (#44291663)
The generous interpretation is that the internet is a communication medium and everyone is just tying g to understand and be understood. The back forth is a continuous effort of clarification of the points being made. The process can break down resulting in no gain in clarity.