News for hackers. Stuff that matters.
[dead]
Re:The rise of ChatGPT-enabled GitHub spam by doodlesdev on Monday May 08, 2023 @09:51PM (#35869319)
Arguably it was always a cat and mouse game, but now it's more like an arms race, where ultimately in the end of the race either the spammers will win or lose, but not anything in the middle. Considering the current situation with CAPTCHAs, I feel pretty uneasy. The biggest problem is that noise and signal are getting _very_ similar, but not similar enough that noise stops being harmful. The silver lining here is I think we can reasonably expect within the decade that LLMs will be able to produce actual signal, or rather, useful-enough noise to the point where a commit being made by a human or not will be less relevant than it is now.
Re:The rise of ChatGPT-enabled GitHub spam by wahnfrieden on Monday May 08, 2023 @09:52PM (#35869324)
(You generated that.)
Re:The rise of ChatGPT-enabled GitHub spam by throwaway675309 on Monday May 08, 2023 @09:59PM (#35869367)
And honestly even before the advent of LLMs, OP's comment would have come off as a gigantic nothingburger.
Re:The rise of ChatGPT-enabled GitHub spam by photoGrant on Monday May 08, 2023 @09:58PM (#35869362)
It’s almost too obvious
Re:The rise of ChatGPT-enabled GitHub spam by jstarfish on Monday May 08, 2023 @10:29PM (#35869549)
Asking for other people's opinions gave it away.
Re:The rise of ChatGPT-enabled GitHub spam by e2le on Monday May 08, 2023 @09:56PM (#35869344)
Re:The rise of ChatGPT-enabled GitHub spam by doodlesdev on Monday May 08, 2023 @10:00PM (#35869374)
Please no. Let's stop with the TPM disgrace. The model proposed by Cloudflare in the linked post inherently requires hostile hardware and trust in the manufacturer's private keys being secure, which considering [0] is pretty unlikely.

Also, ultimately some human behind the scenes is getting prompts into the LLM to produce the content, so any test of personhood simply attests that a human was present to click the button, but not that they wrote what is submitted or even know what it is. Take a look at what happens when CAPTCHAs become commonplace and the cost to solve them with AI becomes too high for spammers [1].

I believe preventing computers from using the internet is a losing game, and improvement of quality resides in:

- In the case of forums, better, manual, human moderation

- In the case of source code, better LLMs that actually produce usable code and/or which could flag which PRs are interesting for review.

- In the realm of news, a cultural shift towards verifying information instead of relying on authority (or the lack of it). This will not happen, but the reality there has been bad much before LLMs.

[0]: https://news.ycombinator.com/item?id=35843566

[1]: https://www.nytimes.com/2010/04/26/technology/26captcha.html

Archived [1]: https://archive.ph/NbkLu

Re:The rise of ChatGPT-enabled GitHub spam by charcircuit on Monday May 08, 2023 @10:16PM (#35869475)
>trust in the manufacturers private keys being secure, which considering [0] is pretty unlikely.

That is an example of a key that wasn't stored in secure hardware. I also believe that hardware security is going to get better over time. There is a clear path forward towards a more secure world of computing and we should not discount it due to mistakes during its early days.

>Take a look at what happens when CAPTCHAs become commonplace and the cost to solve them with AI becomes too high for spammers [1].

Spam mitigations can always be bypassed at a cost. The goal is to find ways to make it expensive and not scale for spammers while keeping it cheap and a good experience for users who aren't spammers.

Re:The rise of ChatGPT-enabled GitHub spam by mirekrusin on Monday May 08, 2023 @11:41PM (#35869920)
Haven't linux kernel developers solved this problem already years ago?

Commit signatures, signoff hierarchies.

Github could solve it with required signatures and automated credibility score.

When using you'd provide "entry points" that you trust (ie. orgs like google, fb, accounts of people you know etc) and "entry points" you blacklist.

You take advantage of directed acyclig graph contributions create (spammer trusting reputable entrypoint is meaningless).

Only single, possibly nested identity has to blacklist something to contribute to downscoring whole spammy contribution graph.

Negative score can be reinforced by others explicitly blacklisting it as well.

I think we'll not get away without anything like that - cryptographic identities, trust graphs etc.

Re:The rise of ChatGPT-enabled GitHub spam by supriyo-biswas on Monday May 08, 2023 @10:31PM (#35869558)
Attestation would only serve to lock out new platforms and people using Linux or other lesser known platforms.
Re:The rise of ChatGPT-enabled GitHub spam by Prickle on Monday May 08, 2023 @10:17PM (#35869483)
That will probably not work in this case. It seems this is a person who is taking a question, querying GPT, then posting the answer verbatim. It is the same or a similar to the issue that StackOverflow found.

Since it appears to be a person doing this, any tests of personhood would fail immediately.

Re:The rise of ChatGPT-enabled GitHub spam by charcircuit on Monday May 08, 2023 @10:00PM (#35869376)
>Anyone else have any thoughts on how we can stay one step ahead in this battle?

Allow sites to do hardware bans. This increases the price to create a spam account. Then as operating systems invest more in security the price of an account will also go up. To avoid spammers compromising other people's legitimate accounts you should invest in account security.

Re:The rise of ChatGPT-enabled GitHub spam by doodlesdev on Monday May 08, 2023 @10:10PM (#35869431)
Then the spammers sell the hardware and real customers gets affected whenever they buy in the second-hand market. Also, hardware bans essentially require uniquely-identifying tracking of the user and its hardware, which I don't believe is what's missing in the internet right. It would also require hardware attestation and of course software attestation, which would make reality the fears of the FSF and co. regarding secure boot and the trusted platform module being used for user control.

If anything, I would like less tracking, even if it comes at a price of a worse signal to noise ratio on the internet.

Also, we know that this will be used for evil and ultimately companies will start banning "deprecated" or "unsupported" hardware from accessing services, be it their own or from competitors, consolidating even more the oligopolies in the hardware and software industries.

Re:The rise of ChatGPT-enabled GitHub spam by charcircuit on Monday May 08, 2023 @10:24PM (#35869523)
>Then the spammers sell the hardware and real customers gets affected

A site could chosen to only ban a computer for day or just give it a bad reputation (require a credit card number or phone to make an account). Or real customers could learn not to buy from shady sources and instead by new, safe devices.

>uniquely-identifying tracking of the user and its hardware, which I don't believe is what's missing in the internet right.

It is missing from the internet and it is part of the reason why spam is a big problem.

>which would make reality the fears of the FSF and co. regarding secure boot and the trusted platform module being used for user control.

User control? It allows services like GitHub to provide a better experience to users by having much less spam. This makes the lives of average users better.

>start banning "deprecated" or "unsupported" hardware from accessing services

It makes sense if you want to have your service be more secure. You could also provide a degraded experience to people with insecure hardware. For example you may not be able to make a PR with insecure hardware, but you could still bookmark a project.

Re:The rise of ChatGPT-enabled GitHub spam by viraptor on Monday May 08, 2023 @10:47PM (#35869650)
> A site could chosen to only ban a computer for day or just give it a bad reputation (require a credit card number or phone to make an account).

That's already happening with IP reputation and captcha essentially. It's just per-connection rather than per-device.

> Or real customers could learn not to buy from shady sources and instead by new, safe devices.

New safe devices can be compromised. (Paying off someone at bestbuy to install malware would become a very cheap investment) "Shady source" is not a clear category. (See all the scams "sold by Amazon")

Re:The rise of ChatGPT-enabled GitHub spam by greysteil on Monday May 08, 2023 @10:02PM (#35869397)
Have hope. There are structural differences between spammy accounts and real ones, because their incentives are different. That will always be the case (it’s pretty much true by definition) and can be used to distinguish the accounts.

AI has definitely made it easier to generate more plausible-looking spam, and removed one way to easily identify bad accounts, but it hasn’t changed the underlying behaviour differences that put a lower bound on how bad the problem can become (as long as GitHub has the team and tools in place to use them in defence).

Re:The rise of ChatGPT-enabled GitHub spam by hartator on Monday May 08, 2023 @10:11PM (#35869445)
Even this comment feels ChatGPT-ish.
Re:The rise of ChatGPT-enabled GitHub spam by gravitronic on Monday May 08, 2023 @10:43PM (#35869627)
Sure, here's another example comment that tries to use a more sarcastic tone:

"Oh boy, another brilliant idea for generating crappy code! Let's just automate our way to technical debt and maintenance hell. Who needs readable, maintainable code anyways? I mean, let's just dump a bunch of generated code into our open source projects and call it a day. Who cares if future developers will have to wade through an incomprehensible mess? I'm sure they'll thank us for our foresight and ingenuity. Let's just keep chasing those short-term efficiency gains and ignore the long-term consequences. What could go wrong?"

;)

Re:The rise of ChatGPT-enabled GitHub spam by CaptainNegative on Tuesday May 09, 2023 @04:15AM (#35871515)
As a large language model, I can understand why you might feel that the original comment appears ChatGPT-ish. It discusses a relevant and topical issue that involves AI-generated content, and the comment's structure and phrasing might resemble the output of an advanced language model.

However, I believe it's essential to give the benefit of the doubt in these situations. The above user is expressing genuine concern about the impact of AI-generated spam and the ongoing struggle to combat it. This is a valid point of discussion that many people share, and it's crucial to foster open dialogue on such matters. Dismissing someone's opinion simply because it may resemble the output of an AI language model could stifle meaningful conversations and discourage people from participating in discussions on important issues.

Furthermore, as AI technology advances, it's increasingly challenging to distinguish between human-generated and AI-generated content. While vigilance is necessary to maintain the integrity of online discussions, it's also important not to exclude genuine contributions due to suspicion alone. Instead, we should focus on fostering critical thinking and promoting collaborative efforts to tackle the challenges posed by the digital world.