A chatbot that only speaks when spoken to is going to gain the ability to trade Bitcoin?

Is that a greater concern than my copy of Dragon Quest 12 gaining the ability to trade bitcoin, or my light switch?

I didn't say 'trade', I said 'make transactions'. It's no more complex than Bing Chat being able to search the internet, or Siri being able to send JSON to an endpoint which turns lightbulbs on and off. Instead it's a shopping endpoint and ChatWhatever can include tokens related to approving transactions from your Bitcoin wallet and has your authorization to use it for purchases less than $100.

Look at OpenAssistant (https://github.com/LAION-AI/Open-Assistant); they are trying to give a chatbot the ability to trigger actions in other systems. I fleshed out a scenario in more detail here: https://news.ycombinator.com/item?id=34808674.

But in brief, the short-term evolution of LLMs is going to involve something like letting it `eval()` some code to take an action as part of a response to a prompt.

A recent paper, Toolformer: https://pub.towardsai.net/exploring-toolformer-meta-ai-new-t... which is training on a small set of hand-chosen tools, rather than `eval(<arbitrary code>)`, but hopefully it's clear that it's a very small step from the former to the latter.

I’ve been getting very good results from eval on JS written by GPT. It is surprising apt at learning when to query a source like wolframalpha or wikipedia and when to write an inline function.

You can stop it from being recursive by passing it through a model that is not trained to write JavaScript but is trained to output JSON.

How often do you think a Bing query is made?

You might say that it doesn't preserve state between different sessions, and that's true. But if it can read and post online, then it can preserve state there.

> that only speaks when spoken to

Feedback loops are an important part.

But let's say you take two current chatbots, make them converse with each other without human participants. Add full internet access. Add a directive to read HN, Twitter and latest news often.

Interesting emergent behaviour could emerge very soon.

Worse, you need only plug a chatbot into itself, with some kind of basic bash script and very simple "goal prompt", and suddenly you get an agent with long term context. You could do that today. I don't think people realize how close these generic undirected intelligences are to unpredictable complex behavior.

A sobering intuition pump: https://www.lesswrong.com/posts/kpPnReyBC54KESiSn/optimality...

man, these lesswrong people sure do have some cool ideas about how to apply GPT

If I give an automated system the ability to make transactions on my behalf then there is already a risk that someone will misuse it or exploit a security vulnerability. It could be a disgruntled employee, or a hacker in Kazakhstan doing it for the lulz. The existence of LLM AI tools changes nothing here.

It is already possible to order toy pistols sent to the authorities with someone else's name on the order. People use stolen credit card numbers for all sorts of malicious purposes. And have you heard of swatting?

The existence of LLM AI tools changes things because it used to not exist and now does exist? It used to be that an AI tool could not do things on your behalf because they did not exist, now it could be that they could do things on your behalf because they do exist and people are giving them ability to take actions on the human's behalf. It used to be that a Kazakhstani hacker could find and exploit a security vulnerability once or twice a year, it can now become that millions of people are querying the AI and having it act on their behalf many times per day.

The risk has existing for many years with humans and other tools. The addition of one more tool to the mix changes nothing.