> I'm not sure of how it works (does it subscribed to them all?) but https://archive.ph/ is a good way to see the content in those cases.

I think for search engine crawlers there are versions without a paywall so these articles can get fully indexed. Archive.ph, and similar services, might get the full content this way somehow. But I am just guessing.

archive.ph also uses (donated) logins to archive (paywalled) content, however those accounts do get blocked from time to time.

https://blog.archive.today/post/678202832257794048/why-cant-...

While pretending to be GoogleBot used to get you full articles (or grabbing them from cache) this doesn't seem to be the case for some sites anymore.

They just give the first part of the article without the paywall, as that's usually enough for SEO purposes.

> They just give the first part of the article without the paywall, as that's usually enough for SEO purposes.

Many consumers often wouldn't read more text anyway. About one paragraph might even be too much to fill the modern attention span.

you're spot on, this is exactly how sites like archive.org, archive.ph, or even if you click on "view cached version" on Google get the non-paywalled versions.

Yes, it's called dynamic rendering when you serve different content based on the user agent, but it's apparently not recommended anymore.

https://searchengineland.com/google-no-longer-recommends-usi...

https://12ft.io is another good one but yes agreed, you should cough up for the subscription when possible

I've tried that on like 3 or 4 different sites and it didn't work for a single one

I noticed too this site is having issues lately. It usually works for me. Either that, or sites have figured out its trick to bypass.

FWIW, I read once that 12ft.io has the business model of charging businesses for the right to stop 12ft.io working on their site.

No, this website, I never succeed even one time...

this.

bloomberg.com for instance, hides pay walled lines in empty <div>s.

the other method is to disable javascript and cookies (works on nytimes.com), or press ESC key to stop page loading before paywall kicks in (works on telegraph.co.uk) :)

> But really, if you are regularly reading content on a site you should subscribe to support the journalists employed there

I'd go further than your statement: I try not to read paywalled contents. Actually I don't get all these workarounds about paywalls. I'm like "they don't want me to read it? I'm not going to read it then".

But they do want you to read. They also want to be financially viable.

Sure, they want me to read if I think they are worth to have my subscription. But I don't think they are worth to have my subscription, otherwise I would subscribe them. So I don't read.

Is this even legal? How can sites host other ones' content, and that's ok?

- Is it fair use because it's "archiving" the web?

- Is it because it's on the open web and it's public domain?

- Or is it illegal, and people do it because they can ¯\_(ツ)_/¯

Is caching illegal?

What archive.ph does is not just caching, it is hosting.

Technically it is illegal. Do I use it? Yes.

How do you "cache" without "hosting"? What is the technical difference between those two acts?

Caching is parroting what the original poster publishes. If the OP asks not to index, you cease indexing. However, if you are purposeful "hosting," you may not accept and continue hosting, knowing all the legal consequences that may follow.

Caching has nothing to do with indexing. If I set up an HTTP cache to serve cached websites on my network, it would be difficult for anyone to request to not cache the HTTP responses considering it is most likely they would not be aware of them at all. It is, however, exactly equivalent to what websites such as archive.ph are doing.

Caching is a generic technology that can be used in lawful or unlawful ways.

Archive.ph et al is run by a russian fellow so probably the third one, especially now that Russia doesn't seem to care about something like this at all

Is Firefox "reader" mode legal?

Is Google cache legal?

What about Internet Archive wayback machine archive?

Is deleting your cookies legal? How about spoofing your user-agent?

How about a browser plugin that automates what OP describes?

In the case of caches like Google, Internet Archive, or `archive.today` (same thing as `archive.ph`)... probably, in the USA? If it winds up in court, we will find out, eventually.

Simply reading anything on the web technically involves "making a copy" already, which is one reason it gets and has remained somewhat confusing and complex to determine what is or is not legal with regard to copying web content. You can't simply say "making a copy is not allowed".

> But really, if you are regularly reading content on a site you should subscribe to support the journalists employed there.

While there are some paywalled websites that allow you to read _n_ articles per period for "free", there are many that don't. How do I know in this case whether it's worth the cost?

There are also times where I'll see a link to something behind a paywall with an interesting headline (frequently on HN), but from a publication I don't regularly read, so have no intention of a subscription. It would be nice in this case to be able to pay a one-time, small contribution.

Worth stating I don't disagree necessarily with the sentiment, there are just a few "edge cases" that make it impractical.

I subscribe to a major newspaper. But I'm not going to subscribe to all major newspapers. The individual subscription model doesn't really fit a world where you can go to one site, and click links to articles from lots of different publications.

If they had a common subscription, where you pay one reasonable fee and they divide it up according to whose articles you read, I'd subscribe to that. Since they don't, I subscribe to one paper and do workarounds on the others. I feel this is ethical because if everyone did it, with a decently random distribution, then the newspapers would survive just fine. They'd make the same overall revenue as when everyone had one newspaper, showing up at their doorstep each morning.

How about getting a paid membership for certain higher education institutions’ libraries, which will give you access to not only books, but also a plethora of periodicals. And that will be fully legal and ethical.

Do you have examples of institutions that do this? A Google search only returned subscriptions to access research papers but not books or periodicals.

Will a library membership remove the hassle when I click a link from reddit?

> If they had a common subscription, where you pay one reasonable fee and they divide it up according to whose articles you read

So some extent that's what Apple News+ (included in Apple One) is. But it doesn't do multi region stuff I don't think, and misses some major publications.

There is a common subscription mostly overlooked. Your local library.

What most of the press subscription services make the mistake of is trying to simplify the billing process to the individual. Ripe opportunity for someone to come along and make an all in one paid subscription service at the county level and make it easy to log into all sites with your library membership.

The current one is per site model is way too much from too not be worth it.

Syndication used to be the way this idea worked. You'd buy your local paper that wrote about the cat stuck in the tree down the street, but it also republished national articles from AP. If you really cared about another locale, you might subscribe to two papers, which meant you might see the same national article twice.

This is also quickly happening with social media now.

Twitter $8, FB $12 (web) / $15 ios)

You're asking for the cable tv model where they aggregate premium channels, imho that doesn't work either... you end up paying for a lot of stuff that you're not interested in.

That is basically the Apple News model. If you have Apple News (part of the Apple One bundle) you can get articles from a number of publishers that would normally require a separate prescription. I’m not sure if the publishers really like this model but so far they seem to be willing to play along.

I stumbled upon a link to an article with an interesting headline. I would like to read it, so I click the link, but there's a paywall. I have no idea what site that even is. This is the first and probably last time I'm seeing it. No way it gets a single cent from me. This just can't work when there are so many news websites competing for subscriptions.

Yes, archive.ph works most of the time, can't recommend it enough.

I’d be more willing to subscribe if publications didn’t pull the dark pattern of making signing up fast and easy, but then requiring a call to a rep to cancel.

I used to subscribe to the nytimes but a few years ago I needed a break from news. My plan was to come back in 6-12 months, but they made me wait on the phone for 25 mins for something that should have taken a couple of minutes on their site. I cancelled and never went back.

I was going to subscribe to NYTdigital .. but I saw the reviews talking about this experience.

Move to the Netherlands. The medium used to sign up, should be available to cancel. Cancellation can't be harder than signing up, and (barring a few exceptions), you can't have an autorenew period of longer than a month, after an initial year.

These may actually be EU rules BTW, haven't checked.

privacy.com

the browser should have a wallet feature, instead of asking you to subscribe, just pay for one article you want to read

You could suspend your subscription. (Shhh – suspending print delivery doesn't interrupt your access to the site or the Replica Edition ツ)

My secret weapon for cutting through the call-to-cancel CSR script with a machete is the following:

"May I ask why you want to cancel your account today?"

"No, you may not."

For some insane reason they ALL phrase the question this way, enabling this little grammatical gem of a response. Enjoy.

Also useful when phoning for many other reasons:

"... calls may be recorded for training purposes."

"Thanks for giving me permission, I am recording this call ..."

+1 for this.

This website almost succeed every time I run out of my tricks, like:

1) ESC to interrupt the page load 2) quickly hit "view mode" before the wall appears 3) add a "." behind the .com, so like .com./ 4) visit in incognito window when the tokens run out (e.g. Medium) 5) Check Google cache of this page, (you can quickly add cache: URL to visit the cache page) 6) Check archive.org cache of some lost pages 7) maybe some extensions but I seldom use them nowadays 8) before, there are some cool sites like, sorry I forgot the names, all stopped working, those websites can remove paypall

9) console tricks though I dunno.