News for hackers. Stuff that matters.
Re:13ft – A site similar to 12ft.io but is self hosted by lutusp on Monday August 19, 2024 @07:39PM (#41295460)
Nice effort, but after one successful NYT session, it fails and treats the access as though it were an end user. But don't take my word for it : try it. One access, succeeds. Two or more ... fails.

The reason is the staff at the NYT appear to be very well versed in the technical tricks people use to gain access.

Re:13ft – A site similar to 12ft.io but is self hosted by echelon on Monday August 19, 2024 @08:38PM (#41295729)
We need a P2P swarm for content. Just like Bittorrent back in the day. Pop in your favorite news article (or paraphrase it yourself), and everyone gets it.

With recommender systems, attention graph modeling, etc. it'd probably be a perfect information ingestion and curation engine. And nobody else could modify the algorithm on my behalf.

Re:13ft – A site similar to 12ft.io but is self hosted by Swarmeggcute on Monday August 19, 2024 @10:26PM (#41296216)
I get the feeling there is some sort of vector of attack behind this idea, but I'm not well versed enough to figure it out.
Re:13ft – A site similar to 12ft.io but is self hosted by bredren on Tuesday August 20, 2024 @03:20AM (#41297639)
I think you have something here.

I don’t know about paraphrased versions but it would need to handle content revisions by the publisher somehow.

Re:13ft – A site similar to 12ft.io but is self hosted by WatchDog on Monday August 19, 2024 @08:43PM (#41295754)
They probably asynchronously verify that the IP address actually belongs to googlebot, then ban the IP when it fails.

Synchronously verifying it, would probably be too slow.

You can verify googlebot authenticity by doing a reverse dns lookup, then checking that reverse dns name resolves correctly to the expected IP address[0].

[0]: https://developers.google.com/search/docs/crawling-indexing/...

Re:13ft – A site similar to 12ft.io but is self hosted by selcuka on Monday August 19, 2024 @09:21PM (#41295962)
> Synchronously verifying it, would probably be too slow.

Why would it be slow? There is a JSON documenbt that lists all IP ranges on the same page you linked to:

https://developers.google.com/static/search/apis/ipranges/go...

Re:13ft – A site similar to 12ft.io but is self hosted by WatchDog on Monday August 19, 2024 @09:40PM (#41296048)
Sure, that's one option, and I don't have any insight into what nyt actually does with regards to it's handling of googlebot traffic.

But if I were implementing filtering, I might prefer a solution that doesn't require keeping a whitelist up to date.

Re:13ft – A site similar to 12ft.io but is self hosted by Gooblebrai on Tuesday August 20, 2024 @02:09AM (#41297243)
The whitelist can be updated asynchronously
Re:13ft – A site similar to 12ft.io but is self hosted by bomewish on Tuesday August 20, 2024 @02:47AM (#41297443)
Maybe we could use GCP infra and the trick will work better ?
Re:13ft – A site similar to 12ft.io but is self hosted by katzgrau on Monday August 19, 2024 @08:50PM (#41295785)
There are easily installable databases of IP block info, super easy to do it synchronously, especially if it’s stored in memory. I run a small group of servers that each have to do it thousands of times per second.
Re:13ft – A site similar to 12ft.io but is self hosted by immibis on Tuesday August 20, 2024 @11:10AM (#41300793)
Which leads to the possibility of triggering a self-inflicted DoS. I am behind a CGNAT right now. You reckon that if I set myself to Googlebot and loaded NYT, they'd ban the entire o2 mobile network in Germany? (or possibly shared infrastructure with Deutsche Telekom - not sure)

Not to mention the possibility of just filling up the banned IP table.

Re:13ft – A site similar to 12ft.io but is self hosted by xp84 on Tuesday August 20, 2024 @11:50PM (#41306617)
Hypothetically if they were doing that, they’d only be ‘banning’ that mobile network in the ‘paywall-relaxing-for-Googlebot’ code - not banning the IP traffic or serving a 403 or anything. They ordinarily throw paywalls at those users anyway.
Re:13ft – A site similar to 12ft.io but is self hosted by noodlesUK on Monday August 19, 2024 @09:27PM (#41295996)
FWIW, if you happen to be based in the U.S., you might find that your local public library provides 3-day NYT subscriptions free of charge, which whilst annoying is probably easier than fighting the paywall. Of course this only applies to the NYT.
Re:13ft – A site similar to 12ft.io but is self hosted by akvadrako on Tuesday August 20, 2024 @02:13AM (#41297277)
In the Netherlands the library provides free access to thousands of newspapers for 5 days after visiting, including The Economist and WSJ, which actually have paywalls that aren't trivial to bypass.

https://www.pressreader.com/

Re:13ft – A site similar to 12ft.io but is self hosted by letier on Tuesday August 20, 2024 @02:32AM (#41297370)
I just checked and the Berlin public library offers press reader as well. Will need to check that out.

Thanks for the tip!

Re:13ft – A site similar to 12ft.io but is self hosted by letier on Tuesday August 20, 2024 @03:37AM (#41297731)
How do you bypass the paywall with it? I can only read the “newspaper version” on their site it seems.

Edit: Just read your comment again. I assume that’s exactly what you meant.

Re:13ft – A site similar to 12ft.io but is self hosted by noodlesUK on Tuesday August 20, 2024 @08:23AM (#41299388)
PressReader allows reading various newspapers (in their newspaper form), the short sub to NYT I mentioned is a bit different and gives you access to the online version.

Example for LA libraries: https://www.lapl.org/new-york-times-digital

Your local library might have a similar offering specifically for the NYT.

Re:13ft – A site similar to 12ft.io but is self hosted by 1vuio0pswjnm7 on Monday August 19, 2024 @10:49PM (#41296311)
"The reason is the staff at the NYT appear to be very well versed in the technical tricks people use to gain access."

It appears anyone can read any new NYT article in the Internet Archive. I use a text-only browser. I am not using Javascript. I do not send a User-Agent header. Don't take my word for it. Here is an example:

https://web.archive.org/web/20240603124838if_/https://www.ny...

If I am not mistaken the NYT recently had their entire private Github repository, a very large one, made public without their consent. This despite the staff at the NYT being "well-versed" in whatever it is the HN commenter thinks they are well versed in.

Re:13ft – A site similar to 12ft.io but is self hosted by markerz on Monday August 19, 2024 @11:10PM (#41296404)
https://securityboulevard.com/2024/08/the-secrets-of-the-new...

Because I had to learn more, sounds like a pretty bad breach. But I’m still pretty impressed by NYTs technical staff for the most part for the things they do accomplish, like the interactive web design of some very complicated data visualizations.

Re:13ft – A site similar to 12ft.io but is self hosted by redxtech on Monday August 19, 2024 @11:47PM (#41296562)
Wasn't the creator/maintainer of svelte on the NYT data visualisation/web technology team?
Re:13ft – A site similar to 12ft.io but is self hosted by stareatgoats on Tuesday August 20, 2024 @02:25AM (#41297345)
Rich Harris, creator of Svelte, worked at the Guardian. Svelte has been adopted at NYT however [0].

You might be thinking of Mike Bostock, the creator of D3.js and ObservableHQ.com, who led data visualization work at NYT for several years [1]. I'm not sure if they have people of that magnitude working for them now.

[0] https://en.wikipedia.org/wiki/Svelte

[1] https://en.wikipedia.org/wiki/Mike_Bostock

Re:13ft – A site similar to 12ft.io but is self hosted by aoeusnth1 on Tuesday August 20, 2024 @02:38AM (#41297400)
Rich did work at the NYT. I thought there was some Mandela effect going on for a second, because you misled me into believing you had actually googled it by providing sources.
Re:13ft – A site similar to 12ft.io but is self hosted by stareatgoats on Tuesday August 20, 2024 @03:49AM (#41297809)
Yeah, my bad. I shouldn't have relied solely on the Wikipedia article and my (sketchy) memory. Rich Harris is still listed as a graphics editor on the investigative team at NYT: https://www.nytimes.com/by/rich-harris
Re:13ft – A site similar to 12ft.io but is self hosted by TeMPOraL on Tuesday August 20, 2024 @03:04AM (#41297533)
Might want to update the Wiki article on Svelte, which strongly implies Rich worked at The Guardian, not NYT. The only source I could quickly find that seems to corroborate what you're saying is a LinkedIn page, but because of its bullshit paywall, there's context missing.
Re:13ft – A site similar to 12ft.io but is self hosted by 91bananas on Tuesday August 20, 2024 @05:02PM (#41304034)
You may be thinking of https://bost.ocks.org/mike/ who worked there for a while, D3 and Observable, among many other things.
Re:13ft – A site similar to 12ft.io but is self hosted by eks391 on Tuesday August 20, 2024 @12:09AM (#41296693)
> I use a text-only browser.

Which browser is this?

Re:13ft – A site similar to 12ft.io but is self hosted by 1vuio0pswjnm7 on Tuesday August 20, 2024 @02:35AM (#41297389)
-4
Re:13ft – A site similar to 12ft.io but is self hosted by shmichael on Tuesday August 20, 2024 @03:01AM (#41297517)
It seems parent is more interested in riddling than informing. Their browser is likely https://en.m.wikipedia.org/wiki/Links_(web_browser)
Re:13ft – A site similar to 12ft.io but is self hosted by pheatherlite on Tuesday August 20, 2024 @04:02AM (#41297868)
When it comes to fruitful discussions that leaves one with satisfaction and contentment, this ain't it. This is the polar opposite. Cheers
Re:13ft – A site similar to 12ft.io but is self hosted by 7bit on Tuesday August 20, 2024 @02:48AM (#41297448)
You could have just said the name in less words facepalm
Re:13ft – A site similar to 12ft.io but is self hosted by Ylpertnodi on Tuesday August 20, 2024 @06:13AM (#41298563)
Perhaps they're a regular contributor to somewhere that discusses 'the browser they use' and don't want any association with it via HN.
Re:13ft – A site similar to 12ft.io but is self hosted by xp84 on Tuesday August 20, 2024 @11:54PM (#41306633)
I would hope that they’re in IA on purpose. It would be exceptionally lame if NYT didn’t let their articles be archived by the Internet’s best and most definitive archive. It would be scary to me if they had only been allowing IA archiving because they were too stupid to know about it.
Re:13ft – A site similar to 12ft.io but is self hosted by roshankhan28 on Tuesday August 20, 2024 @03:08AM (#41297560)
i tried to access my own website and it says internal server error. i also tried to access Youtube and it said the same.
Re:13ft – A site similar to 12ft.io but is self hosted by lodovic on Tuesday August 20, 2024 @03:20AM (#41297641)
Some sites do work, but others such as WSJ just give a blank page. Worse, Economist actively blocks this through Cloudflare.