News for hackers. Stuff that matters.
Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by seanp2k2 on Monday June 05, 2017 @04:05PM (#14491199)
This. I installed the personal blacklist extension for Chrome just to blacklist Quora results because of this. I wish Google would actually punish them more. I click through on something which looks relevant, and the result is a useless login screen. LinkedIn is the same.
Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by pkkm on Monday June 05, 2017 @04:08PM (#14491260)
You can bypass the Quora login screen by appending ?share=1 to the URL. There's also an userscript which does this automatically: https://github.com/sindresorhus/quora-unblocker-userscript
Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by devrandomguy on Monday June 05, 2017 @04:46PM (#14491814)
If you change your browser's UserAgent string to Googlebot, then your client will be treated as a first-class citizen, by many of these sites. Google always wins, so let's all be Google.
Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by Xeoncross on Monday June 05, 2017 @04:58PM (#14491964)
Works great until you stumble on one of the big sites who will auto-ban you for not having a valid Google IP address.
Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by Spivak on Monday June 05, 2017 @05:55PM (#14492551)
* Reverse DNS records. Webmasters shouldn't be verifying Google's bots by hard coding IP addresses.

https://support.google.com/webmasters/answer/80553?hl=en

Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by imron on Monday June 05, 2017 @06:49PM (#14492998)
Shouldn't != never happens
Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by dredmorbius on Monday June 05, 2017 @06:51PM (#14493012)
CIDR blocks and ASN advertisments are cheap.

Update those periodically (hours / days / weeks). The adverts don't change particularly quickly.

Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by dmn001 on Monday June 05, 2017 @05:58PM (#14492588)
It's extremely rare to be ip-blocked by any website just for using the Google's user agent from a non-specific range. IP's get re-used and you can switch to a new one easily, so it's really not common or good practice for this to happen.
Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by kbenson on Monday June 05, 2017 @09:32PM (#14494025)
> IP's get re-used and you can switch to a new one easily, so it's really not common or good practice for this to happen.

On the flip side, some people can't change their IP addresses easily, and getting IP banned (even if rare because of the reasons you stated) is actually a major hassle when it actually happens for those people. :/

Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by devrandomguy on Monday June 05, 2017 @05:07PM (#14492060)
Is that really a thing? That must be such a hazard for their developers. I usually have a test for sites that I work on, that scrapes a few URLs as Googlebot, to verify that they are getting an optimized view (no JS, structural-only css).
Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by Normal_gaussian on Monday June 05, 2017 @05:34PM (#14492328)
God. That's the reason so many sites look great in the results and are confusing interactive messes when I get into them.
Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by devrandomguy on Monday June 05, 2017 @05:37PM (#14492363)
If it's any consolation, the site is well tested in Noscript mode.
Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by SomeCallMeTim on Monday June 05, 2017 @06:01PM (#14492605)
You sir are a god among Web developers.

If only they all did this. So many sites I get to and they're a blank page or an absolute disaster....

Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by ChuckMcM on Monday June 05, 2017 @05:36PM (#14492349)
Yes. Googlebot only crawls from legit addresses (even when their developers are trying new things) so it's an easy scraper/scammer signal to key off of.
Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by dmn001 on Monday June 05, 2017 @06:03PM (#14492623)
No. Most websites don't do this.
Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by ipsum2 on Monday June 05, 2017 @05:22PM (#14492226)
I just tried this, doesn't work on WSJ. Tried the user agents listed here: https://support.google.com/webmasters/answer/1061943?hl=en
Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by bduerst on Monday June 05, 2017 @05:36PM (#14492348)
Isn't that what the entire article is about? That WSJ no longer gives access to Google users (and bots)?
Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by mirimir on Monday June 05, 2017 @07:43PM (#14493385)
No. It allows Google bots to see full articles, but shows only the first paragraph or so to non-subscribers. Even if they're coming from Google search results.

However, I don't see cache links on Google :(

Edit: Oops, I'm wrong. The article does say that the Google bot only sees the first paragraph or so.

Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by Jimmie_Rustle on Monday June 05, 2017 @08:01PM (#14493512)
No. You're wrong, as it states in the article:

"The reason: Google search results are based on an algorithm that scans the internet for free content. After the Journal’s free articles went behind a paywall, Google’s bot only saw the first few paragraphs and started ranking them lower, limiting the Journal’s viewership."

Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by mirimir on Monday June 05, 2017 @09:31PM (#14494024)
Yes, you're right. I got confused by all the discussion about Google checking for cloaking by comparing results using different user agents.

So maybe this is why there's no Google cache.

Also, if Google can only index the first few paragraphs, the results are much less comprehensive.

Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by __jal on Monday June 05, 2017 @05:18PM (#14492185)
Or just turn off Javascript. Makes lots of sites better.
Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by cynwoody on Tuesday June 06, 2017 @01:31AM (#14494976)
I call that enacting the nuclear option. It's almost guaranteed to win the war with ad-tech! It should be enacted for sites with run-away ad engines that spin up your CPU fans and make scrolling laggy.

Of course, the problem with nuclear is collateral damage. Drop the bomb and ads don't work, but neither does a lot of other stuff. E.g., the site shows a blank screen, images are invisible or blurry, drop-down menus don't drop. And, of course, the deal-breaker: videos don't play.

The remedy for killing JavaScript is more JavaScript (and CSS). But supplied inside a Chrome extension targeted at the offending site. An injected stylesheet makes `<body>` visible again, hides assorted useless junk, and styles injected UI elements. Your content scripts load the missing images, drop the menus down, and play the unplayable videos in button-activated pop-over windows displayed at superior resolution.

Of course, the problem is, there are a lot of sites out there, and they change unpredictably, requiring your extension library to change in response. That argues for crowd-sourcing the extension library, but the crowd needs to be proficient in HTML, JavaScript, and CSS and know the ins and outs of browser extensions and care and have time.

You can completely change how a site presents. E.g., change a slide-show in a static slide window that barely moves due to the background ad-tech load changes into a set of `divs` that roll upwards as your finger swipes.

It's a hobby at best. Disabling ad-tech components by origin is the practical option.

Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by __jal on Tuesday June 06, 2017 @05:11PM (#14501380)
Call me Dr. Strangelove, then. I usually browse with JS off, enabling it on occasion. And there are some whitelisted sites.

I used to play around with filtering sites to make them less antisocial, but find that slog less entertaining these days. So now when confronted with a site that's useless without JS, eh, there's almost always another site out there that doesn't mind the terms I demand for my attention.

Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by kevmo on Monday June 05, 2017 @04:30PM (#14491590)
Ah, I made a similar (but simpler) one years ago: https://github.com/kevmo/greasemonkey/blob/master/quora_upvo...
Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by amelius on Monday June 05, 2017 @04:27PM (#14491559)
Pinterest too.
Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by derekp7 on Monday June 05, 2017 @04:57PM (#14491954)
The worst thing about Pinterest results, is they aren't primary sites -- I'd rather get a link back to the source page the image came from.
Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by jquery on Monday June 05, 2017 @06:42PM (#14492945)
You could always use Pinterest directly if that's what you want.
Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by et-al on Monday June 05, 2017 @06:47PM (#14492986)
Do you need an account to actually make Pinterest useful?

Sometimes I do a Google Image search because I found something interesting, but don't know what it is, so I'm hoping of going to a page that describes what I was looking at. Pinterest shows up as a result, but with no backstory nor does it lead me back to a source, so it's worthless as a search result. It'd the ExpertsExchange of image searches.

Endy's advice of using `-inurl:pinterest` seems invaluable and I'll be adding that for all image searches in the future.

Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by jquery on Monday June 05, 2017 @07:05PM (#14493107)
You can use Pinterest directly without signing up - https://www.pinterest.com/search/
Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by nyolfen on Monday June 05, 2017 @05:14PM (#14492131)
pinterest has ruined image search results, it's infuriating
Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by pkamb on Monday June 05, 2017 @05:51PM (#14492507)
Would kill for a Google Image Search extension that forwards my browser to the original source page rather than the Pinterest landing page. Any recommendations?
Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by Papirola on Monday June 05, 2017 @06:00PM (#14492603)
would you pay for it?
Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by pkamb on Monday June 05, 2017 @06:39PM (#14492917)
One-time payment of a few bucks, sure.
Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by anodari on Monday June 05, 2017 @05:01PM (#14491997)
Agree. Pinterest search results is spam.
Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by Endy on Monday June 05, 2017 @05:52PM (#14492512)
You know, even in image searches "-inurl:pinterest" works pretty well.
Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by dawnerd on Monday June 05, 2017 @04:40PM (#14491740)
Google images feel like it's nothing but Pinterest now.
Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by shostack on Monday June 05, 2017 @05:20PM (#14492197)
Frankly I'd love Google to turn Google Images into much more of a true Pinterest competitor. Image search volume is pretty large, and they are already testing monetizing it. Now they just need to add in features that let you store and organize that information (and build a better profile of you in the process while also crowdsourcing tagging).

Pinterest needs more competition.

Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by jquery on Monday June 05, 2017 @06:39PM (#14492925)
> X needs more competition.

In the context of Google being the alternative, this is sad and funny at the same time.

Hacker News in the abstract: The web needs more decentralization.

Hacker News IRL: Let Google own every vertical.

Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by shostack on Monday June 05, 2017 @07:25PM (#14493244)
I certainly recognize the irony of suggesting that Google might help improve competition.

That said, there aren't many players in a position to provide actual competition to Pinterest like Google can. The obvious concern is where you draw the line at anti-competitive if Google tried to do the equivalent of what they did with Yelp ratings.

Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by jlarocco on Monday June 05, 2017 @06:46PM (#14492975)
If they did that I would switch to Bing's image search immediately. Definitely not what I want when I'm searching for an image.
Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by sverige on Monday June 05, 2017 @06:20PM (#14492768)
Bing is a lot better than Google for searching images. It also has better tools built in to the results page (similar images, different sizes, multiple source pages).
Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by mistermann on Monday June 05, 2017 @06:27PM (#14492819)
Could you PLEASE PLEASE PLEASE link to this extension so I can install it to, I used to have one but lost it and couldn't find one that worked last time I checked. (Would love one for both Chrome and FF if it isn't cross platform).

Would also love to have one that rewrites URL's in search results to avoid the frequent 5 second pause when Google's redirector gets it's head stuck up its ass or whatever the problem is.

Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by arkitaip on Monday June 05, 2017 @09:28PM (#14494012)
Personal Blocklist by Google https://chrome.google.com/webstore/detail/personal-blocklist...

Note that it doesn't block some spam domains that sneakily use certain special characters i their domain names. Unfortunately Google hasn't fixed this issue for forever.

Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by mistermann on Tuesday June 06, 2017 @12:48AM (#14494807)
Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by senorjazz on Monday June 05, 2017 @05:09PM (#14492082)
Have never signed up to quora and I don't get these nag screens :-s
Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by hoschicz on Monday June 05, 2017 @05:27PM (#14492271)
What's your problem with Quora? They provide great answers most of their time and their digests are very inspiring.
Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by kbos87 on Monday June 05, 2017 @05:33PM (#14492319)
Quora is one of the biggest offenders of growth tactics at the cost of user experience. They make it so easy to create an account without intending to that I probably have 15 of them.
Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by trendia on Monday June 05, 2017 @05:28PM (#14492279)
The Google search result looks like you'll see the content when you open the link, but instead you have to login first. OP doesn't want to login.
Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by Jedd on Monday June 05, 2017 @06:32PM (#14492865)
This 2013 article sums up nicely why quora is frustrating to be directed to while seeking enlightenment, and as others have pointed out they use some unpleasant anti-patterns to lure in (potential) users.

http://jesuschristsiliconvalley-blog.tumblr.com/post/4896203...