Makes me wonder if creating a plugin that makes your browser pretend to be the Google indexing bot would give you secret access to all paysites?

Certainly at this point anybody serious about wanting to give Google special access through their paywall would allow based on the published IP blocks [1] and not an easily spoofed UA header

[1] https://developers.google.com/search/docs/crawling-indexing/...

I remember the good old days when Google penalized sites for showing content to their crawler that wasn't available to normal users.

https://12ft.io works on some sites through pretending to be the google bot.

You can also access any site in the google cache with prepending "https://webcache.googleusercontent.com/search?q=cache:", that will you show you the website like the google bot saw it.

For example github.com would become "https://webcache.googleusercontent.com/search?q=cache:https%..."

It is still worth to try, but many sites already prevent this.

It just recently stopped working for Zeit.de articles. Seems like their paywall is now higher than 12 feet...