News for hackers. Stuff that matters.
Re:AI companies cause most of traffic on forums by alphan0n on Monday December 30, 2024 @08:47PM (#42555570)
I would take anything the author said with a grain of salt. They straight up lied about the configuration of the robots.txt file.

https://news.ycombinator.com/item?id=42551628

Re:AI companies cause most of traffic on forums by mplewis on Tuesday December 31, 2024 @07:39PM (#42563001)
What is causing you to be so unnecessarily aggressive?
Re:AI companies cause most of traffic on forums by alphan0n on Tuesday December 31, 2024 @08:57PM (#42563372)
Liars should be called out, necessarily. Intellectual dishonesty is cancer. I could be more aggressive if it were something that really mattered.
Re:AI companies cause most of traffic on forums by nkrisc on Tuesday December 31, 2024 @09:51PM (#42563585)
Lying requires intent to deceive. How have you determined their intent?
Re:AI companies cause most of traffic on forums by n144q on Tuesday December 31, 2024 @10:34PM (#42563774)
> Lying requires intent to deceive

Since when do we ask people to guess other people's intent when they have better things to show, which is called evidence?

Surely we should talk about things with substantiated matter?

Re:AI companies cause most of traffic on forums by nkrisc on Wednesday January 01, 2025 @08:41AM (#42565949)
Because there’s a meaningful difference between being wrong and lying.

There’s evidence the statement was false, no evidence it was a lie.

Re:AI companies cause most of traffic on forums by alphan0n on Tuesday December 31, 2024 @10:48PM (#42563827)
When someone says:

> Oh, and of course, they don't just crawl a page once and then move on. Oh, no, they come back every 6 hours because lol why not. They also don't give a single flying fuck about robots.txt, because why should they.

Their self righteous indignation and specificity of the pretend subject of that indignation precludes any doubt about intent.

This guy made a whole public statement that is verifiably false. And then tried to toddler logic it away when he got called out.

Re:AI companies cause most of traffic on forums by nkrisc on Wednesday January 01, 2025 @08:41AM (#42565944)
That may all be true. That still doesn’t mean they intentionally lied.
Re:AI companies cause most of traffic on forums by alphan0n on Wednesday January 01, 2025 @04:51PM (#42569607)
What is the criteria of an intentional lie, then? Admission?

The author responded:

>denschub 2 days ago [–]

>the robots.txt on the wiki is no longer what it was when the bot accessed it. primarily because I clean up my stuff afterwards, and the history is now completely inaccessible to non-authenticated users, so there's no need to maintain my custom robots.txt

Which is verifiably untrue:

HTTP/1.1 200 server: nginx/1.27.2 date: Tue, 10 Dec 2024 13:37:20 GMT content-type: text/plain last-modified: Fri, 13 Sep 2024 18:52:00 GMT etag: W/"1c-62204b7e88e25" alt-svc: h3=":443", h2=":443" X-Crawler-content-encoding: gzip Content-Length: 28

User-agent: * Disallow: /w/

Re:AI companies cause most of traffic on forums by nkrisc on Friday January 03, 2025 @07:09AM (#42584946)
> intentional lie

There are no “intentional” lies, because there are no “unintentional” lies.

All lies are intentional. An “unintentional lie” is better known as “being wrong”.

Being wrong isn’t always lying. What’s so hard about this? An example:

My wife once asked me if I had taken the trash out to the curb, and I said I had. This was demonstrably false, anyone could see I had not. Yet for whatever reason, I mistakenly believed that I had done it. I did not lie to her. I really believed I had done it. I was wrong.

Re:AI companies cause most of traffic on forums by alphan0n on Saturday January 04, 2025 @02:09AM (#42593069)
No worries, I understand. The author admitted to me that he was lying via DM, that he often does this for attention.
Re:AI companies cause most of traffic on forums by ribadeo on Wednesday January 01, 2025 @12:01PM (#42567297)
How do you know what the contextual configuration of their robots.txt is/was?

Your accusation was directly addressed by the author in a comment on the original post, IIRC

i find your attitude as expressed here to be problematic in many ways

Re:AI companies cause most of traffic on forums by alphan0n on Wednesday January 01, 2025 @04:37PM (#42569521)
CommonCrawl archives robots.txt

For convenience, you can view the extracted data here:

https://pastebin.com/VSHMTThJ

You are welcome to verify for yourself by searching for “wiki.diasporafoundation.org/robots.txt” in the CommonCrawl index here:

https://index.commoncrawl.org/

The index contains a file name that you can append to the CommonCrawl url to download the archive and view.

More detailed information on downloading archives here:

https://commoncrawl.org/get-started

From September to December, the robots.txt at wiki.diasporafoundation.org contained this, and only this:

>User-agent: * >Disallow: /w/

Apologies for my attitude, I find defenders of the dishonest in the face of clear evidence even more problematic.

Re:AI companies cause most of traffic on forums by shkkmo on Thursday January 02, 2025 @09:19AM (#42574617)
Your attitude is inappropriate and violates the sitewide guidelines for discussion.
Re:AI companies cause most of traffic on forums by alphan0n on Friday January 03, 2025 @01:50AM (#42583213)
There are currently two references to “Mangion-ing” OpenAI board members in this thread, several more from Reddit, based on the falsehoods being perpetrated by the author. Is this really someone you want to conspire with? Is calling this out more egregious than the witch hunt being organized here?
Re:AI companies cause most of traffic on forums by shkkmo on Friday January 03, 2025 @11:53AM (#42587233)
"conspire" and "witch hunt", are not terms of productive discourse.

If you are legitimately trying to correct misinformation, your attitude, tone and language are counter productive. You would be much better seved by taking that energy and crafting an actually persuasive argument. You come across as unreasonable and unwilling to listen, not someone with a good grasp of the technical specifics.

I don't have a horse in the race. I'm fairly technical, but I did not find your arguments persuasive. This doesn't mean they are wrong, but it does mean that you didn't do a good job of explaining them.