News for hackers. Stuff that matters.
Re:WormGPT – The Generative AI Tool Cybercriminals Are Using by brucethemoose2 on Saturday July 15, 2023 @10:05PM (#36742822)
> GPT-J is the LLM, the old one from 2021

Thats very interesting.

The infamous Pygmalion 6B is a GPT-J finetune, predating the LLM craze. Yet its decent in its roleplaying niche.

But the LLaMA 13B version, with instruct finetuning, is massively better, even with dataset errors that allegedly messed up its performance. In fact, a chat with Metharme 13b, where it made some very introspective logical jumps, was my first real LLM "Wow!" moment.

And Airoboros-Chronos 33b is leagues ahead of that.

If someone in that forum has a 3090, and trains LLaMA 33b on that dataset + a instruct dataset off huggingface... Yeah, that would be terrifying.

Re:WormGPT – The Generative AI Tool Cybercriminals Are Using by KRAKRISMOTT on Saturday July 15, 2023 @10:08PM (#36742839)
Phishing emails self select, I don't think generative ML would make a difference. Those "typos" are sometimes intentional. The scammers are not illiterate. They are more than capable of using Grammarly if they wanted to.
Re:WormGPT – The Generative AI Tool Cybercriminals Are Using by brucethemoose2 on Saturday July 15, 2023 @10:21PM (#36742902)
But what if emails can be mass produced at 15+ words/second, on a gaming PC?

And what if they can very convincingly and quickly reply?

Re:WormGPT – The Generative AI Tool Cybercriminals Are Using by 6177c40f on Saturday July 15, 2023 @10:44PM (#36743019)
A bunch of random thoughts:

Why would being able to write different emails help? Wouldn't they just reuse the same email over and over again, maybe just changing the addressee's name?

Also, now I wonder what the typical response rate is for phishing emails? Are they already getting too many responses to keep up with or are they already managing just fine without bots?

I think LLMs might be better for scams carried out over instant messaging services like Discord because that probably requires greater attention and flexibility on the part of the scammer.

Still, has the number of people that could conceivably be scammed over an email or instant messaging-based scam increased? That is, will LLMs, by virtue of their response quality, responsiveness, or some other salient quality, increased the pool of people who could fall for these scams? If the answer is no, then I don't see how increased scale and throughput for human-quality text generation will really matter much.

Re:WormGPT – The Generative AI Tool Cybercriminals Are Using by jsight on Saturday July 15, 2023 @10:50PM (#36743043)
My guess is that being able to generate lots of different text helps to bypass existing spam and phishing protections.

Think of it as a large scale A/B test of circumvention techniques against both human and automated scam detection systems.

Done right, it can effectively keep getting better on its own. Then add phone scamming and various other interactive scams once it is good enough. TBH, it is pretty scary where this will inevitably head.

Re:WormGPT – The Generative AI Tool Cybercriminals Are Using by userbinator on Sunday July 16, 2023 @01:03AM (#36743853)
And what if they can very convincingly and quickly reply?

Turn this around, and have an LLM that replies to scammers and keeps them busy.

There's already https://en.wikipedia.org/wiki/Lenny_(bot) , but this is an area where AI could actually be a useful addition.

Re:WormGPT – The Generative AI Tool Cybercriminals Are Using by nwiswell on Sunday July 16, 2023 @07:01AM (#36745575)
That gets pretty awkward the first time that a LLM turns out to have intentionally stalled a critical business process.
Re:WormGPT – The Generative AI Tool Cybercriminals Are Using by fnordpiglet on Saturday July 15, 2023 @11:09PM (#36743145)
My understanding, which can’t source the reference for off hand, is there is an idea that if someone can detect it’s a phishing attack from a typo they are too smart to fall for the overarching scam, hence the intentionality.
Re:WormGPT – The Generative AI Tool Cybercriminals Are Using by jokethrowaway on Sunday July 16, 2023 @07:25AM (#36745746)
I think there is some truth to that and some actual illiteracy going on
Re:WormGPT – The Generative AI Tool Cybercriminals Are Using by mozman on Sunday July 16, 2023 @01:04AM (#36743860)
I have never heard of any of these models before. How does one stay up to date?
Re:WormGPT – The Generative AI Tool Cybercriminals Are Using by brucethemoose2 on Sunday July 16, 2023 @01:14AM (#36743900)
Look at models on huggingface, sorted by date, with keywords like "65b" (for llama 65b) or "ggml" (aka a pre quantized model for llama.cpp inference).

https://huggingface.co/models?sort=modified&search=65b+ggml

Note that 33b llama models show up under "30b" or "33b" due to an old typo from Meta.

I also look at what people are choosing to host here, though Kobold is mostly centered around the roleplaying crowd: https://lite.koboldai.net/#

Re:WormGPT – The Generative AI Tool Cybercriminals Are Using by tak2hu on Sunday July 16, 2023 @02:42AM (#36744263)
Re:WormGPT – The Generative AI Tool Cybercriminals Are Using by flangola7 on Sunday July 16, 2023 @05:09AM (#36744986)
>How does one stay up to date?

Have you tried using an AI to monitor and summarize new updates?

Re:WormGPT – The Generative AI Tool Cybercriminals Are Using by flangola7 on Sunday July 16, 2023 @05:06AM (#36744969)
There's still 65B as well.