News for hackers. Stuff that matters.
Re:Ask HN: What are these low quality “code snippet” sites? by MrWiffles on Wednesday December 01, 2021 @12:37PM (#29406650)
Well I think you already nailed it with “not even trying”. You don’t need to try or optimize when you’re a monopoly. Competition forces that function, but a lack thereof enables mediocrity and complacency.
Re:Ask HN: What are these low quality “code snippet” sites? by kristiandupont on Wednesday December 01, 2021 @12:51PM (#29406821)
Oh come on! I am no fan of Google and their privacy policies etc., and I agree that results like the sites mentioned in this thread are annoying but I am not so cocky as to think I have any idea what Google is fighting.

There are a lot of very smart people from all over the world putting everything they have into getting to the top of that site. It's not a trivial task for them.

Re:Ask HN: What are these low quality “code snippet” sites? by Nextgrid on Wednesday December 01, 2021 @03:54PM (#29409014)
I disagree. Google has absolutely dropped the ball - I'm not saying that it's easy to algorithmically rank good/bad content, but they're not even trying.

A stop-gap solution would be to simply penalize anything with ads. Legitimate websites will still have enough "SEO juice" (for the lack of a better term) to offset the penalty, but brand new copycats with otherwise no inbound links to them from other legitimate websites (and no other business model beyond scammy ads) will never be able to rank high, essentially removing the incentive from setting up these sites in the first place.

Not to mention, these problems can be identified, prioritized and then tackled manually one-by-one. Stackoverflow copycats can be dealt with by downloading the SO data dump, parsing it and then severely penalizing any website where the bulk of the content matches the dump. Pinterest can be penalized by simply excluding it from image searches until they actually display the searched image without asking for login. So on and so forth.

You can't win them all, and you can't train an algorithm to win every time either, but you can manually observe what's happening and deal with the biggest problems.

The problem however is that the current status-quo is good for Google. They've got the monopoly on search (every other one is typically even worse when it comes to these issues), and spammy copycat websites happen to have Google ads or analytics so Google actually benefits from them too.

Re:Ask HN: What are these low quality “code snippet” sites? by TheOtherHobbes on Wednesday December 01, 2021 @09:51PM (#29411949)
Well, quite. Why would a company that makes its money selling ads penalise sites that show its ads?

And when you get to that state, you've pretty much turned the entire web into content marketing for your ad sales business.

So - of course search quality suffers. Search quality is not the point.

Re:Ask HN: What are these low quality “code snippet” sites? by Kiro on Wednesday December 01, 2021 @05:30PM (#29410003)
How do you know Google is not already doing all those things? You said it yourself, you can't win them all and we're witnessing the result of that. If Google truly did nothing it would be an unimaginable chaos.
Re:Ask HN: What are these low quality “code snippet” sites? by Nextgrid on Wednesday December 01, 2021 @05:54PM (#29410214)
Because I’m not seeing the results. Let’s ignore SO copycats for now because that solution requires at least a slight amount of effort and look at the elephant in the room - Pinterest. Dealing with it is a simple “if image_search and domain == pinterest.com then skip” and they’re not even doing that.

They have a site that’s been polluting image search results for years, isn’t even trying to hide (unlike the SO copycats which could technically churn out an infinite amount of domains to work around bans) and they can’t even deal with that.

Re:Ask HN: What are these low quality “code snippet” sites? by MrWiffles on Wednesday December 01, 2021 @04:22PM (#29409293)
Consider it from a management/budgetary point of view. Super smart people? Oh hell yeah, and I don't fault them for this one bit. But bean counters? Why the hell invest in hiring those people, or budgeting for search quality improvements, when you have essentially no competition to speak of? Makes more sense to put those same smart people on projects that correlate people's activities outside of Google with activities they can observe either directly or indirectly to create a more wholistic picture of the consumer for advertisers. Those are much harder problems to solve, especially now that the public is finally becoming slowly more aware of the massive abuses of privacy the bean counters like I described above have been pushing for years.

At the end of the day Google doesn't exist for the good of the public, they exist to return the highest possible investment for shareholders. It's literally the opposite of their job to improve search quality when competition doesn't force them to (because investors see that as a waste of money that could have been spent on further eroding people's privacy to enable higher and higher ad revenues.)

Re:Ask HN: What are these low quality “code snippet” sites? by kizer on Wednesday December 01, 2021 @02:11PM (#29407837)
Fair point. But again, that would be less of an issue if there were like 3 or 4 big search engines with different algorithms, all changing and adapting on their own schedules. Would be harder to game the system then.
Re:Ask HN: What are these low quality “code snippet” sites? by motoxpro on Wednesday December 01, 2021 @02:11PM (#29407841)
You're so right. It's sad that even if you remove 99% of something, to the outside it looks like you are removing 0% because you have no reference.
Re:Ask HN: What are these low quality “code snippet” sites? by rhdunn on Wednesday December 01, 2021 @01:04PM (#29406989)
One of the problems with things like this is how do you know which site has copied from another, especially if you don't want a list of hard-coded exceptions.

Related is if you have a lot of content from git repositories that are mirrored from different locations (GitHub, GitLab, etc.), all of which are showing the same content. Or if different sites are hosting versions of public domain texts. You don't want to derank those results, even if they are similar to the "copy a popular site" websites.

Re:Ask HN: What are these low quality “code snippet” sites? by MrWiffles on Wednesday December 01, 2021 @04:25PM (#29409332)
> One of the problems with things like this is how do you know which site has copied from another, especially if you don't want a list of hard-coded exceptions.

The core function of that is actually pretty simple:

1. Strip all X/HTML tags 2. Run `diff`

Sure, it's not perfect, but an organization that pursues academic quantum computing research can sure as hell afford to run the results of the above against an AI to check for similarities.

Re:Ask HN: What are these low quality “code snippet” sites? by rhdunn on Thursday December 02, 2021 @03:18AM (#29413714)
The issue is not figuring out if site X has a copy of a page or text from site Y. The issue is how do you know which site is the source of the information. In other words, the technical bit is easy but determining what to do with that is hard. Especially if you don't want a lot of hard coded exceptions.

In the case of things like other sites copying from stack overflow, github, etc. you can figure out which is the source of the information.

Lets day you have decided to make github the source of that information, and derank any other sites that also have that information. As a result, you will derank gitlab for having git mirrors of projects, as the source will match that on github. You will also derank sites like lkml as that contains the commit message descriptions, and the patches will partially match content from the kernel source hosted on github.

Lets also say you've decided to make wikipedia and other wikimedia sites the source of information. Congratulations, you've now deranked project gutenberg for hosting the same public domain texts as wikimedia, along with other sites like the official sites for authors like Jack London. Plus any blog that includes portions of these to discuss or analyze them.

Do you want a blackbox AI making these decisions?

Re:Ask HN: What are these low quality “code snippet” sites? by skinkestek on Thursday December 02, 2021 @04:42AM (#29414220)
> he issue is how do you know which site is the source of the information. In other words, the technical bit is easy but determining what to do with that is hard.

I interviewed with a 3 or so person company that did just that I think.

That was over a decade ago.

Re:Ask HN: What are these low quality “code snippet” sites? by lazyeye on Wednesday December 01, 2021 @02:05PM (#29407772)
Google knows instantly how much time you spend on a page and how quickly you return to the search results because it is a useless copy/paste page from SO.
Re:Ask HN: What are these low quality “code snippet” sites? by cutemonster on Wednesday December 01, 2021 @03:07PM (#29408512)
And probably they track content age too? Since they somehow can lookup by content (they can detect duplicated content), there's probably a date remembered too

So they could know if a code snippet was already X years old? And from where, originally. (Unless it got edited a lot but that's not the case here?)