News for hackers. Stuff that matters.
Re:Ask HN: What are these low quality “code snippet” sites? by stevesimmons on Wednesday December 01, 2021 @12:29PM (#29406517)
> they seem to be losing the content junk spam SEO fight

Google clearly isn't even trying.

So many of these sites are polluting search results for months. It isn't a case of sites that pop up for a few hours until Google notices and blacklists them.

Google Search has gone so far downhill. I'm not sure what they're optimising for. Long-term irrelevance, it seems.

Re:Ask HN: What are these low quality “code snippet” sites? by MrWiffles on Wednesday December 01, 2021 @12:37PM (#29406650)
Well I think you already nailed it with “not even trying”. You don’t need to try or optimize when you’re a monopoly. Competition forces that function, but a lack thereof enables mediocrity and complacency.
Re:Ask HN: What are these low quality “code snippet” sites? by kristiandupont on Wednesday December 01, 2021 @12:51PM (#29406821)
Oh come on! I am no fan of Google and their privacy policies etc., and I agree that results like the sites mentioned in this thread are annoying but I am not so cocky as to think I have any idea what Google is fighting.

There are a lot of very smart people from all over the world putting everything they have into getting to the top of that site. It's not a trivial task for them.

Re:Ask HN: What are these low quality “code snippet” sites? by Nextgrid on Wednesday December 01, 2021 @03:54PM (#29409014)
I disagree. Google has absolutely dropped the ball - I'm not saying that it's easy to algorithmically rank good/bad content, but they're not even trying.

A stop-gap solution would be to simply penalize anything with ads. Legitimate websites will still have enough "SEO juice" (for the lack of a better term) to offset the penalty, but brand new copycats with otherwise no inbound links to them from other legitimate websites (and no other business model beyond scammy ads) will never be able to rank high, essentially removing the incentive from setting up these sites in the first place.

Not to mention, these problems can be identified, prioritized and then tackled manually one-by-one. Stackoverflow copycats can be dealt with by downloading the SO data dump, parsing it and then severely penalizing any website where the bulk of the content matches the dump. Pinterest can be penalized by simply excluding it from image searches until they actually display the searched image without asking for login. So on and so forth.

You can't win them all, and you can't train an algorithm to win every time either, but you can manually observe what's happening and deal with the biggest problems.

The problem however is that the current status-quo is good for Google. They've got the monopoly on search (every other one is typically even worse when it comes to these issues), and spammy copycat websites happen to have Google ads or analytics so Google actually benefits from them too.

Re:Ask HN: What are these low quality “code snippet” sites? by TheOtherHobbes on Wednesday December 01, 2021 @09:51PM (#29411949)
Well, quite. Why would a company that makes its money selling ads penalise sites that show its ads?

And when you get to that state, you've pretty much turned the entire web into content marketing for your ad sales business.

So - of course search quality suffers. Search quality is not the point.

Re:Ask HN: What are these low quality “code snippet” sites? by Kiro on Wednesday December 01, 2021 @05:30PM (#29410003)
How do you know Google is not already doing all those things? You said it yourself, you can't win them all and we're witnessing the result of that. If Google truly did nothing it would be an unimaginable chaos.
Re:Ask HN: What are these low quality “code snippet” sites? by Nextgrid on Wednesday December 01, 2021 @05:54PM (#29410214)
Because I’m not seeing the results. Let’s ignore SO copycats for now because that solution requires at least a slight amount of effort and look at the elephant in the room - Pinterest. Dealing with it is a simple “if image_search and domain == pinterest.com then skip” and they’re not even doing that.

They have a site that’s been polluting image search results for years, isn’t even trying to hide (unlike the SO copycats which could technically churn out an infinite amount of domains to work around bans) and they can’t even deal with that.

Re:Ask HN: What are these low quality “code snippet” sites? by MrWiffles on Wednesday December 01, 2021 @04:22PM (#29409293)
Consider it from a management/budgetary point of view. Super smart people? Oh hell yeah, and I don't fault them for this one bit. But bean counters? Why the hell invest in hiring those people, or budgeting for search quality improvements, when you have essentially no competition to speak of? Makes more sense to put those same smart people on projects that correlate people's activities outside of Google with activities they can observe either directly or indirectly to create a more wholistic picture of the consumer for advertisers. Those are much harder problems to solve, especially now that the public is finally becoming slowly more aware of the massive abuses of privacy the bean counters like I described above have been pushing for years.

At the end of the day Google doesn't exist for the good of the public, they exist to return the highest possible investment for shareholders. It's literally the opposite of their job to improve search quality when competition doesn't force them to (because investors see that as a waste of money that could have been spent on further eroding people's privacy to enable higher and higher ad revenues.)

Re:Ask HN: What are these low quality “code snippet” sites? by kizer on Wednesday December 01, 2021 @02:11PM (#29407837)
Fair point. But again, that would be less of an issue if there were like 3 or 4 big search engines with different algorithms, all changing and adapting on their own schedules. Would be harder to game the system then.
Re:Ask HN: What are these low quality “code snippet” sites? by motoxpro on Wednesday December 01, 2021 @02:11PM (#29407841)
You're so right. It's sad that even if you remove 99% of something, to the outside it looks like you are removing 0% because you have no reference.
Re:Ask HN: What are these low quality “code snippet” sites? by rhdunn on Wednesday December 01, 2021 @01:04PM (#29406989)
One of the problems with things like this is how do you know which site has copied from another, especially if you don't want a list of hard-coded exceptions.

Related is if you have a lot of content from git repositories that are mirrored from different locations (GitHub, GitLab, etc.), all of which are showing the same content. Or if different sites are hosting versions of public domain texts. You don't want to derank those results, even if they are similar to the "copy a popular site" websites.

Re:Ask HN: What are these low quality “code snippet” sites? by MrWiffles on Wednesday December 01, 2021 @04:25PM (#29409332)
> One of the problems with things like this is how do you know which site has copied from another, especially if you don't want a list of hard-coded exceptions.

The core function of that is actually pretty simple:

1. Strip all X/HTML tags 2. Run `diff`

Sure, it's not perfect, but an organization that pursues academic quantum computing research can sure as hell afford to run the results of the above against an AI to check for similarities.

Re:Ask HN: What are these low quality “code snippet” sites? by rhdunn on Thursday December 02, 2021 @03:18AM (#29413714)
The issue is not figuring out if site X has a copy of a page or text from site Y. The issue is how do you know which site is the source of the information. In other words, the technical bit is easy but determining what to do with that is hard. Especially if you don't want a lot of hard coded exceptions.

In the case of things like other sites copying from stack overflow, github, etc. you can figure out which is the source of the information.

Lets day you have decided to make github the source of that information, and derank any other sites that also have that information. As a result, you will derank gitlab for having git mirrors of projects, as the source will match that on github. You will also derank sites like lkml as that contains the commit message descriptions, and the patches will partially match content from the kernel source hosted on github.

Lets also say you've decided to make wikipedia and other wikimedia sites the source of information. Congratulations, you've now deranked project gutenberg for hosting the same public domain texts as wikimedia, along with other sites like the official sites for authors like Jack London. Plus any blog that includes portions of these to discuss or analyze them.

Do you want a blackbox AI making these decisions?

Re:Ask HN: What are these low quality “code snippet” sites? by skinkestek on Thursday December 02, 2021 @04:42AM (#29414220)
> he issue is how do you know which site is the source of the information. In other words, the technical bit is easy but determining what to do with that is hard.

I interviewed with a 3 or so person company that did just that I think.

That was over a decade ago.

Re:Ask HN: What are these low quality “code snippet” sites? by lazyeye on Wednesday December 01, 2021 @02:05PM (#29407772)
Google knows instantly how much time you spend on a page and how quickly you return to the search results because it is a useless copy/paste page from SO.
Re:Ask HN: What are these low quality “code snippet” sites? by cutemonster on Wednesday December 01, 2021 @03:07PM (#29408512)
And probably they track content age too? Since they somehow can lookup by content (they can detect duplicated content), there's probably a date remembered too

So they could know if a code snippet was already X years old? And from where, originally. (Unless it got edited a lot but that's not the case here?)

Re:Ask HN: What are these low quality “code snippet” sites? by testudovictoria on Wednesday December 01, 2021 @03:56PM (#29409030)
> Google Search has gone so far downhill. I'm not sure what they're optimising for. Long-term irrelevance, it seems.

I assume they're optimizing for the most common denominator: standard, non-power users. Early search didn't return great results for human-like questions such as What are the wavelengths of the colors on the visible spectrum? The result might be within the first two pages, but that query had too many irrelevant search terms. A better query would have been wavelengths color visible spectrum. That query only has the necessary key terms. Sometimes queries required the user to know search operators (e.g. exact match, date range, synonyms) just to get relevant results.

The average person probably didn't know that early searches gave better results when constructed in the second way. Google changed search to adapt to how normal people search. Now the human-like query will return good results. Combine that with locales, search history, and personal interests, even the most basic user can get worthwhile results from asking Google question. The cost is that power users who understand operators and the power of key terms get less relevant results but likely still correct.

Re:Ask HN: What are these low quality “code snippet” sites? by acdha on Wednesday December 01, 2021 @07:27PM (#29411018)
I'm skeptical that it's all that much better for any group of user. I search for normal things all of the time and it doesn't do a good job at returning relevant results for things like businesses, recent movies / TV shows, etc. and natural language syntax doesn't make the results less bad.

What it looks suspiciously like to me is a lack of an effective feedback loop for user frustration — if it takes a number of queries to get correct results or someone doesn't stop using Google search entirely, this would be easy to confuse with improved engagement and I'd especially believe that managers whose job it is to get a number to go up are not in a hurry to question whether that growth is meaningful.

Re:Ask HN: What are these low quality “code snippet” sites? by stevesimmons on Wednesday December 01, 2021 @04:45PM (#29409543)
If Google actually used its search history, it would see I was a sw eng power user, and return the types of results that users like me want to see.

Then again, I have been using Google Maps for 16 years and it still cannot show me distances in km rather than miles. A boolean setting ffs that I've had to manually switch probably a thousand times.

If billions of dollars of AI R&D can't even figure that out...

Re:Ask HN: What are these low quality “code snippet” sites? by Kiro on Wednesday December 01, 2021 @05:34PM (#29410040)
Funny how people complain about Google tracking them and then get angry when they are not being tracked enough.
Re:Ask HN: What are these low quality “code snippet” sites? by gwittel on Wednesday December 01, 2021 @01:57PM (#29407648)
Pretty much this. It’s likely down to the lack of incentives for leaders and line workers. Ie: search spam volume isn’t a measure by which search team performance is graded
Re:Ask HN: What are these low quality “code snippet” sites? by vadfa on Wednesday December 01, 2021 @01:05PM (#29407003)
Is it legally a good idea for Google to go and blacklist individual sites?
Re:Ask HN: What are these low quality “code snippet” sites? by kevin_thibedeau on Wednesday December 01, 2021 @02:38PM (#29408177)
They killed off the Wikipedia clones that way. It's funny that they haven't come back even though the SO clones do the exact same thing.
Re:Ask HN: What are these low quality “code snippet” sites? by SrslyJosh on Wednesday December 01, 2021 @02:43PM (#29408231)
Wikipedia clones probably got enough traffic to show up on someone's dashboard/report. SO clones are probably a small fraction of all traffic, even though they seem to show up in a large percentage of the searches we perform.
Re:Ask HN: What are these low quality “code snippet” sites? by skinkestek on Thursday December 02, 2021 @04:43AM (#29414231)
I guess Wikipedia kind of got their rampant deletionism under control while Stack Exchange did not?
Re:Ask HN: What are these low quality “code snippet” sites? by GhettoComputers on Wednesday December 01, 2021 @01:31PM (#29407318)
Child porn is the extreme. www.chillingeffects.org is the other.
Re:Ask HN: What are these low quality “code snippet” sites? by briffle on Wednesday December 01, 2021 @02:15PM (#29407894)
Exactly, change the word blacklist to censor, and then it seems wrong.
Re:Ask HN: What are these low quality “code snippet” sites? by SrslyJosh on Wednesday December 01, 2021 @02:41PM (#29408204)
But that's not censorship. Google is not a government, and no one's speech is being suppressed by filtering out literal copies of something and leaving the original.
Re:Ask HN: What are these low quality “code snippet” sites? by snerbles on Wednesday December 01, 2021 @03:15PM (#29408615)
Censorship can be conducted by private or public entities.

https://www.aclu.org/other/what-censorship

Re:Ask HN: What are these low quality “code snippet” sites? by labster on Wednesday December 01, 2021 @02:38PM (#29408174)
Change the flag button on this post to censor and I bet people wouldn’t click it. But people want some level of censorship, so long as they convince themselves they’re not against free speech.
Re:Ask HN: What are these low quality “code snippet” sites? by SrslyJosh on Wednesday December 01, 2021 @02:44PM (#29408246)
Blacklisting by a private entity is not censorship. It's even sillier when you consider that what's being removed are copies and the original content is still there.
Re:Ask HN: What are these low quality “code snippet” sites? by Nasrudith on Wednesday December 01, 2021 @03:41PM (#29408878)
Isn't that a bit narcissistic to think that your one problem in particular out of millions is a sign that they aren't doing anything at all?