The LLM is continuing to communicate with the bot because that is literally all an LLM can do -- predict the next sequence of tokens.

Yes - tokens. Which aren't necessarily conversation responses - i.e. it can predict it should cease communication, and output whatever it's been told will terminate it (perhaps by invoking a tool).

No, it can refuse to talk by outputting an <eos> token at any point if it predicts that there is nothing more to be said.

Technically still "just a token" yes, but it does flow control instead.

Of course, that its function. It is able to refuse to continue conversing by restating its refusal over and over, though.

My immediate thought at the start of this article was not DoS but more about harming the company using the chatbot (Company A) by increasing their chatbot bills. In many (most?) cases they will not be hosting their chatbot and will instead be getting it from a 3rd party provider (Company B) who may not even be truly hosting it either.

If the pricing structure is per conversation or per month it would harm Company B, but not the likely target, Company A. If it is paid per interaction it would harm Company A and benefit Company B who just get more paid work.

It feels a bit like cases of rivals clicking on each other's ads to cost them on ad spend, but presumably much lower value than ads.

You would think it would be easy to stop a conversation at n interactions via some other means than relying on the LLM itself, but then you also have to figure out how to stop the attacker just starting more conversations (or passing the output of one of your chatbot instances into the input of another)

If costs of outlier conversations are a concern but any party, they can just end the conversation after 1,000 or 10,000 responses or whatever. What human would ever reach that threshold? Surely no customer worth keeping, whatever you’re selling.

That is easy to solve. Just use a model capable of function/tool calling, implement a tool which terminates the chat, then add instructions to the system prompt telling the model what tool to use if it wants to end the conversation. If the model appears too hesitant or eager to use the tool, do some finetuning on conversations where the model should or should not use it.

LLMs aren’t capable of “comprehending” anything. They never “know” what they are outputting, they’re simply really good at being lucky. They are not lucky enough to be useful unless you’re already an expert on the topic you’re using them on so that you can spot when they aren’t lucky.

This is part of why many enterprise organisations are banning their usage. It’s one thing to use them to build software poorly, the world is already used to IT not working very often. It’s another thing to produce something that has real world consequences. Our legal department used them in a PoC for contract work, and while they were useful very often they also sometimes got things very wrong. Unlike a slow IT system, this would have business shattering consequences. You can continue training your model as well as reigning it in when it gets unlucky, but ultimately you can never be sure it’s never unlucky, and this means that LLMs are useless for a lot of things. We still use them to make pretty PowerPoint presentations and so on, but again, this is an area where faults are tolerable.

I don't believe that current models have the capability to 'comprehend', I was using the term loosely. However I find that people tend to go to extremes when they want to make the point that language models are not 'intelligent' by minimizing their capability and complexity behind 'it is just math', which I think is unhelpful because it merely acts as a 'though-terminating cliche'.

I think the issue is far more psychological than technical personally. One of the issues we struggle with for our junior developers is that they are far more likely to believe an LLM than what they might find Google programming. I do wonder why we’ve ended up with CS graduates who go to LLMs and Search engines before the official documentation which is often very, very good, but I guess that’s a different discussion.

I’m not personally against LLM assistance, I use it for programming and it has in many places replaced my usage of snippets completely. This is probably why I’m not really a fan of the “knowledge” part that LLMs are increasingly tasked to do. Because when you use them for programming you’ll get an accrue insight into how terrible they can be when they get things wrong.

At this point I think what is happening some people either have a natural inclination or they spend time to learn how to use them productively, while others question their usefulness. People scoff at 'prompt engineering' but I see how some of my peers use LLMs and I think to myself 'how do they expect to get a good answer from that?'

It doesn't help that google is now mostly full of SEO nonsense, and technical documentation is impenetrable when you are looking for something specific but don't know enough about the system to know how to look for it.

You and I are a mix of molecules arranged in a particular way that responds to electrical, physical, and chemical inputs.

It’s entirely possible than an LLM will do something that can be defined as “comprehending” something.

Doesn't mean LLM architecture is complex enough. Clearly it's far from the ultimate human invention.

> It’s entirely possible than an LLM will do something that can be defined as “comprehending” something.

Agreed, but not even necessary. Each Human uses an adequately customized meaning of "comprehending" when they "comprehend" this problem space such that their belief is always "true". This is how Humans are able to produce numerous "true" statements that disagree with each other.

If you disagree, just ask one of them and they will "inform" you of the "logic" behind their version.

Joking aside though: if Humans are unable to comprehend past a certain level of complexity, I wonder what will happen once AI starts going beyond our abilities, both at the individual level and the culturally conditioned ("the" "reality") level.

And, when is this going to really start kicking in?

>LLMs aren’t capable of “comprehending” anything. They never “know” what they are outputting, they’re simply really good at being lucky.

The mental gymnastics people will go through to discount LLMs is wild. This does not even make any sense.

"really good at being lucky". What does that even mean ?

> "really good at being lucky". What does that even mean ?

They mean good at being lucky the way card counters playing 21 are good at being lucky.

There is almost nothing 'lucky' about how good those kind of players are.