News for hackers. Stuff that matters.
Re:uBlacklist: Blocks specific sites from appearing in Googl... by dopa42365 on Monday December 13, 2021 @08:41PM (#29547145)
Some basic filters for stuff that pops up all the time and is never what I'm looking for:

/.pinterest.\../

/.torrent.\../

/.spankbang.\../

/.mp3.\../

/.proxy.\../

/.block.\../

/.addiction.\../

/.recovery.\../

/.detox.\../

/.clinic.\../

/.rehab.\../

/.rarbg.\../

/.hydra.\../

/.123.\../

/.stock.\../

/.linkedin.\../

/.subtitle.\../

/.shopify.\../

/.dreamstime.\../

/.depositphotos.\../

/^(m|mobile|it|es|ja|fr|pl|at|ru)\.\w./

Also recommend TLD filters:

/.\.(tk|photo|dev|page|science|ninja|monster|st|cricket|store|website|help|bg|party|work|xyz|link|camera|computer|click|cc|club|pw|space|info|rocks|review|faith|date|win|racing|site|online|tech|news|life|design|nyc|guru|photography|global|today|solutions|media|world|biz|gq|rest|ga|casa|buzz|cam|ml|fit|ly|it|vip|php|pl|ru|app|name|tr|asia|surf|top|cf|tw|uz|am|krd|is|cz|pt|pk|cl|il|in|hu|hr|si|ee|bid|vn|kw|ro|mx|me|market|nu|how|red|guide|mobi|cafe|tn|xxx|icu|fun|pro|desi|id|live|one|ir|forsale)\/(.*)/

(modify for your personal needs, it'll save you many, many hours in the long term)

Re:uBlacklist: Blocks specific sites from appearing in Googl... by jagger27 on Monday December 13, 2021 @09:47PM (#29547612)
Ah yes the notoriously spammy .dev TLD.

I guarantee you that .com, .net, and .org have far more spam and low quality sites than this stupid list.

Re:uBlacklist: Blocks specific sites from appearing in Googl... by dredmorbius on Tuesday December 14, 2021 @02:49AM (#29549285)
The original generic TLDs have many highly-ranked legitimate sites.

The problem with extended generic TLDs is that the spam sites have managed to become highly ranked. Yes, there's good stuff there, undoubtedly. But search engines can't find it. At which point it may as well not exist.

I'm not a fan of outright blocking at this level without good cause, but it could well be worthwhile to block an entire TLD in cases.

[dead]
Re:uBlacklist: Blocks specific sites from appearing in Googl... by rewq4321 on Monday December 13, 2021 @11:24PM (#29548223)
Re:uBlacklist: Blocks specific sites from appearing in Googl... by altantiprocrast on Monday December 13, 2021 @11:08PM (#29548124)
Or golang at go.dev
Re:uBlacklist: Blocks specific sites from appearing in Googl... by emodendroket on Tuesday December 14, 2021 @01:05AM (#29548737)
I'll add that if you're getting unwanted ads for that many drug addiction recovery-related terms constantly, perhaps your habits do merit examination.
Re:uBlacklist: Blocks specific sites from appearing in Googl... by viraptor on Monday December 13, 2021 @08:54PM (#29547231)
You're losing lots of quality content by removing those TLDs. I'm using 3 of the listed ones just because they're cheaper and found similar ones to be a good indicator of "this is less likely to be blogspam, and more likely a personal space".
Re:uBlacklist: Blocks specific sites from appearing in Googl... by sodapopcan on Monday December 13, 2021 @09:47PM (#29547611)
Ya, I use them for personal space. My last name is five letters ending in "st" and I managed to snag that three-letter st domain which I was pretty happy about :) I know a lot of people seem to hate them--and a lot of non-tech-types don't always seem to understand that they are domains--but I think they're fun, and I love finding peoples' personal homepages (mine is currently borked, of course, but not because of the st domain).
Re:uBlacklist: Blocks specific sites from appearing in Googl... by dspillett on Tuesday December 14, 2021 @06:15AM (#29550448)
There is also a lot of junk on those TLDs, especially the inexpensive ones, outweighing the ratio of good-to-crap found on the big few.

Blocking them in search lists doesn't stop you finding them indirectly by them being mentioned in content elsewhere, or from even more direct referrals, so the non-junk ones can be found those ways. I tend to find useful content on .com et al as much those ways as by direct search these days anyway.

Re:uBlacklist: Blocks specific sites from appearing in Googl... by Debug_Overload on Monday December 13, 2021 @08:56PM (#29547243)
I'm not sure how spankbang got on that list but it's hilarious. Is it really a common problem like pinterest and similar stuff on that list?
Re:uBlacklist: Blocks specific sites from appearing in Googl... by _fat_santa on Monday December 13, 2021 @09:01PM (#29547276)
If there is such thing as being "too good" at SEO, Pinterest is "too good" at SEO. It's a common complaint with image search that Pinterest will cannibalize the results. What will often happen is someone will repost an image from a website or blog to Pinterest and then when you look for images of that thing, you get the image that was reposted to Pinterest instead of the OG image.
Re:uBlacklist: Blocks specific sites from appearing in Googl... by thaumasiotes on Monday December 13, 2021 @09:31PM (#29547495)
It's supposed to be an adversarial system. Why does Google still provide Pinterest search results, given that if you follow them, you're not allowed to see the image?
Re:uBlacklist: Blocks specific sites from appearing in Googl... by jquery on Monday December 13, 2021 @09:35PM (#29547522)
I doubt Google wants to, but their users keep clicking on Pinterest's SEO'd results for "cute cake pop" instead of whatever non-Pinterest result Google serves up. Basically Google being outdone at their own game in this niche area.
Re:uBlacklist: Blocks specific sites from appearing in Googl... by thaumasiotes on Monday December 13, 2021 @09:36PM (#29547528)
Google special-cases their own stuff all the time. Why haven't they done that for Pinterest?
Re:uBlacklist: Blocks specific sites from appearing in Googl... by jquery on Monday December 13, 2021 @09:45PM (#29547591)
Who says they haven't? As to why they haven't totally blocked Pinterest, that would be pretty controversial given Pinterest paints inside the lines.

edit: oh, I see the confusion. People misunderstand what Google image search is, how it works, and are blaming Pinterest for Google's ineptitude.

Re:uBlacklist: Blocks specific sites from appearing in Googl... by chongli on Monday December 13, 2021 @10:48PM (#29547996)
People will correct me if I’m wrong, but I believe the practice of showing an image in search results and then hiding that image behind a login wall when users click the link would qualify as cloaking [1], a practice Google specifically prohibits [2].

[1] https://en.wikipedia.org/wiki/Cloaking

[2] https://developers.google.com/search/docs/advanced/guideline...

Re:uBlacklist: Blocks specific sites from appearing in Googl... by dspillett on Tuesday December 14, 2021 @06:27AM (#29550526)
Unfortunately this and similar rules are not well enforced. The only high-profile case I can remember was when they delisted BMW for a short while for blatant keyword stuffing.
Re:uBlacklist: Blocks specific sites from appearing in Googl... by thaumasiotes on Monday December 13, 2021 @09:51PM (#29547628)
> given Pinterest paints inside the lines.

Do they? They advertise images in their search results that you aren't allowed to see when you follow the link. It's some of the most paradigmatic black-hat behavior there is.

Re:uBlacklist: Blocks specific sites from appearing in Googl... by jquery on Monday December 13, 2021 @11:00PM (#29548065)
You can blame Google for that, how on earth is that Pinterest's fault? Google is the one choosing an arbitrary thumbnail out of hundreds of images to represent "pinterest.com/cake-pops". Then Google uses that thumbnail to refer people to Pinterest, but doesn't tell Pinterest what thumbnail they used. Pinterest's hands are tied here. This is Google's fault, 100%.
Re:uBlacklist: Blocks specific sites from appearing in Googl... by margalabargala on Monday December 13, 2021 @11:20PM (#29548203)
Well, considering that a user can't view the image, Pinterest is showing images to Google differently than to the average user. What Pinterest does by hiding the images is specifically against Google's policy. There's no question that Pinterest deliberately went out of their way to create the currently existing paradigm, to suggest their hands are tied and they are somehow a passive victim of Google rather than an actively malicious actor is ludicrous.

It may well be the case that this is Google's fault and they are a malicious actor here as well as Pinterest, but Pinterest is blatantly malicious and does not draw within the lines.

Re:uBlacklist: Blocks specific sites from appearing in Googl... by jquery on Friday December 24, 2021 @09:24AM (#29673797)
You're seeing the same images that Google does, it's just that these are dynamic pages that change over time. Google might only crawler pinterest.com/cute-cupcakes/ every day, for example. Meanwhile, Pinterest refreshes the content of that page every day, but at a different time than Google crawls it. Boom, you click on the image google showed for "/cute-cupcakes/" and what Pinterest was showing is no longer there to be found. Both Google and Pinterest are misbehaving here. Google is using Pinterest to beef up its image search. Pinterest is using Google to get traffic. But Google won't share any of its precious data with other companies unless they pay for it, so Google doesn't pass anything like a hash or X-ORIGINAL-IMAGE-URL in the referrer headers when you click on a link, so Pinterest's hands are tied, sort of.

Anyway plenty of bad behavior to go around. Both Pinterest or Google could probably solve the issue with better engineering. I put a bit more responsibility on Pinterest, but I'm not giving Google a total pass here either.

Re:uBlacklist: Blocks specific sites from appearing in Googl... by jquery on Monday December 13, 2021 @09:33PM (#29547507)
This happens because Pinterest is better at categorizing images than Google is. Google should just buy Pinterest and integrate them directly into their image search. Or partner with them so this doesn't happen. Or they could do what Bing did and just copy Pinterest feature by feature.
Re:uBlacklist: Blocks specific sites from appearing in Googl... by thetis on Tuesday December 14, 2021 @05:26AM (#29550188)
interesting, does that mean that there is a part of 'active' pinterest users who take time to add tags to pics, letting pinterest to categorize a lot of pictures and let the google ranking higher?
Re:uBlacklist: Blocks specific sites from appearing in Googl... by skocznymroczny on Tuesday December 14, 2021 @03:11AM (#29549402)
At the moment it's one of my favorite websites of "this type", because they haven't done the content purge yet like the main sites did.
Re:uBlacklist: Blocks specific sites from appearing in Googl... by Gigachad on Monday December 13, 2021 @11:04PM (#29548093)
That's a whole lot of TLDs to block. .me is very common for personal pages/blogs/emails. .app is used for a lot of legitimate sites as well.
Re:uBlacklist: Blocks specific sites from appearing in Googl... by fennecfoxen on Monday December 13, 2021 @08:48PM (#29547194)
dot NYC is full of spam? A pity. Hard to get voting or ferry info here without it. :)
Re:uBlacklist: Blocks specific sites from appearing in Googl... by dspillett on Tuesday December 14, 2021 @06:34AM (#29550566)
Then edit the list instead of blindly using it? And surely you know the local addresses for such things, or do you search for them using Google every single time you need to travel or vote?
Re:uBlacklist: Blocks specific sites from appearing in Googl... by ouid on Monday December 13, 2021 @08:49PM (#29547201)
God damn it. I don't want heroin detox; I want heroin.
Re:uBlacklist: Blocks specific sites from appearing in Googl... by jean_valje4n on Monday December 13, 2021 @09:03PM (#29547294)
Careful what you wish for
Re:uBlacklist: Blocks specific sites from appearing in Googl... by jquery on Monday December 13, 2021 @09:39PM (#29547543)
you really don't...
Re:uBlacklist: Blocks specific sites from appearing in Googl... by DrSiemer on Tuesday December 14, 2021 @03:45AM (#29549594)
Is there a word or expression for when people fail to spot fairly obvious sarcasm?
Re:uBlacklist: Blocks specific sites from appearing in Googl... by alvarlagerlof on Monday December 13, 2021 @10:54PM (#29548036)
.dev blocks a lot of personal blogs
Re:uBlacklist: Blocks specific sites from appearing in Googl... by Gigachad on Monday December 13, 2021 @11:05PM (#29548101)
Almost all personal blogs fall under that tld filter.
Re:uBlacklist: Blocks specific sites from appearing in Googl... by Otek on Tuesday December 14, 2021 @05:32AM (#29550211)
This "recommended" TLD list would block my bank site, government tax service and most of news sited since my country TLD is blocked here. Hilarious.
Re:uBlacklist: Blocks specific sites from appearing in Googl... by dspillett on Tuesday December 14, 2021 @06:32AM (#29550554)
It is only a suggested list. Edit as needed depending on locality. I don't expect it would inconvenience you if it blocked my local bank or government tax service.

Always review blocklists rather than applying them automatically as your needs may vary from their creators'. No one list is going to be right for everyone, so criticising one for not being right for you is a bit "me me me me".

Re:uBlacklist: Blocks specific sites from appearing in Googl... by dspillett on Tuesday December 14, 2021 @06:10AM (#29550430)
You need to escape your * characters, by prepending them with \ like \*, to stop HN interpreting them as markup intended to control italic/not.
Re:uBlacklist: Blocks specific sites from appearing in Googl... by ipiz0618 on Monday December 13, 2021 @10:53PM (#29548031)
I'm using .xyz domains for my side projects because they are cheap. Guess they are because they look "spammy" to some.
Re:uBlacklist: Blocks specific sites from appearing in Googl... by jjeaff on Tuesday December 14, 2021 @02:59PM (#29556703)
Interesting thing about .XYZ is that if you try to send someone a link with a .xyz url via SMS it usually just gets quietly spam filtered out by the carriers and never received.
Re:uBlacklist: Blocks specific sites from appearing in Googl... by guerrilla on Monday December 13, 2021 @09:55PM (#29547659)
Finally, death to pinterest!