Ask HN: Let's build an HN uBlacklist to improve our Google search results?

Re:Ask HN: Let's build an HN uBlacklist to improve our Googl... by nixcraft on Tuesday January 04, 2022 @11:21AM (#29796505)

May I know why my domain (cyberciti.biz) was added to that list? I created my site back in 2000, and there was no StackOverflow or anything. So much for creating original content and then getting labelled as a spammer. In fact, some of the top answers on StackOverflow were copied from my work without giving any credit to me. Some people do give credit tho. But, go ahead block a site that actual humans maintain over 20+ years. Also check my About[1] and Twitter[2] page. There is no scrapping or spamming going on my part.

[1]https://www.cyberciti.biz/tips/about-us [2]https://twitter.com/nixcraft

[ Parent | on HN ]

Re:Ask HN: Let's build an HN uBlacklist to improve our Googl... by mikevin on Tuesday January 04, 2022 @02:23PM (#29798889)

Interesting, I have your site on my mental blocklist as one of those scrape and rehost sites.

I'll be honest, I don't remember how I came to that conclusion but I suspect I encountered an unsatisfactory answer to a question I was looking to answer, saw the .biz and drew my conclusions.

The noise to signal ratio for most of my queries is so high that I have to start judging a book by its title, not even its cover.

[ Parent | on HN ]

Re:Ask HN: Let's build an HN uBlacklist to improve our Googl... by Karsteski on Tuesday January 04, 2022 @01:39PM (#29798291)

I've noticed cyberciti.biz showing up in my DDG search results but I've always ignored it because of the initial captcha. I will try it now that I've seen your post here!

The .biz definitely does not help, since it hints to me that it's just another one of those worthless reposting sites, as someone else commented below.

[ Parent | on HN ]

Re:Ask HN: Let's build an HN uBlacklist to improve our Googl... by travisporter on Tuesday January 04, 2022 @11:48AM (#29796846)

Not OP but dot biz is associated with spam in my head for what it’s worth

[ Parent | on HN ]

Re:Ask HN: Let's build an HN uBlacklist to improve our Googl... by nixcraft on Tuesday January 04, 2022 @11:59AM (#29796961)

A couple of years ago, I was at Google's office, and I talked with someone who works on search about .biz extension. They said domain extension doesn't matter. At that time, they said backlinks is one of the most vital signals apart from some PR. That was like eight years ago. So I never changed the domain name despite owning the .com version too. It will break too many backlinks.

[ Parent | on HN ]

Re:Ask HN: Let's build an HN uBlacklist to improve our Googl... by lacksconfidence on Tuesday January 04, 2022 @12:13PM (#29797142)

Sure, google might be fine with a .biz. As a human consuming googles responses, my eyes typically glaze over seeing .biz and jump to the next search result. It's not that there is anything particularly wrong with .biz, but this is the first legitimately useful site (to me, probably plenty for others) i've heard of using .biz.

[ Parent | on HN ]

Re:Ask HN: Let's build an HN uBlacklist to improve our Googl... by zxexz on Tuesday January 04, 2022 @11:36AM (#29796693)

cyberciti.biz is one of the few sites that come up in Google search results for anything code/linux related that has valuable content. I do wonder why someone would block it.

[ Parent | on HN ]

Re:Ask HN: Let's build an HN uBlacklist to improve our Googl... by taxyz23 on Wednesday January 05, 2022 @10:02AM (#29809794)

I agree. Valuable site and nixcraft adds a ton of value to the linux community. Thank you nixcraft!

[ Parent | on HN ]

Re:Ask HN: Let's build an HN uBlacklist to improve our Googl... by nixcraft on Wednesday January 05, 2022 @03:15AM (#29806443)

Thank you for your support!

[ Parent | on HN ]

Re:Ask HN: Let's build an HN uBlacklist to improve our Googl... by pbowyer on Tuesday January 04, 2022 @04:57PM (#29801155)

I wanted to stop by and say thanks for cyberciti.biz! I've been using it since 2001-2002 when I got my first Verio Freebsd VPS and had to figure out what was going on.

When I see your site pop up in my search results I know the content is going to be more reliable than most of the others. Thanks for the effort you've put into it.

[ Parent | on HN ]

Re:Ask HN: Let's build an HN uBlacklist to improve our Googl... by burnished on Tuesday January 04, 2022 @11:50AM (#29796856)

At first scan your site looks like one of those automated scrape and republish sites. I'm curious what got you on that blacklist (misspelling? bad first impression? automated tool gone awry?) though.

Glad you said something though, I wouldn't have looked at it twice without a human attestation.

[ Parent | on HN ]

Re:Ask HN: Let's build an HN uBlacklist to improve our Googl... by nixcraft on Tuesday January 04, 2022 @12:00PM (#29796986)

I kept it simple on purpose. As a result, it loads faster on both desktop and mobile and passes web.dev PageSpeed insight test too.

[ Parent | on HN ]

Re:Ask HN: Let's build an HN uBlacklist to improve our Googl... by GreenWatermelon on Tuesday January 04, 2022 @03:08PM (#29799612)

For me personally, the titles are what makes it suspicious for me. I've almost never, NEVER found something good in an article titled "(top) xy ways to z", I've come to immediately avoid any article with such a title.

[ Parent | on HN ]

Re:Ask HN: Let's build an HN uBlacklist to improve our Googl... by burnished on Wednesday January 05, 2022 @04:55AM (#29807114)

Yep, and it doesn't break if you have javascript disabled. Good work.

That doesn't really intersect with my observation in any way though. As a stranger I don't see your intent when I see your website, I just see your website.

[ Parent | on HN ]

Re:Ask HN: Let's build an HN uBlacklist to improve our Googl... by jodrellblank on Tuesday January 04, 2022 @08:05PM (#29803487)

I'm always happy to see your site in search results, it's one I recognise and trust for CentOS/Linux related information for years. Thank you!

[ Parent | on HN ]

Re:Ask HN: Let's build an HN uBlacklist to improve our Googl... by nixcraft on Wednesday January 05, 2022 @03:15AM (#29806447)

Thanks for the kind words!

[ Parent | on HN ]

Re:Ask HN: Let's build an HN uBlacklist to improve our Googl... by endisneigh on Tuesday January 04, 2022 @01:25PM (#29798096)

Your comment is exactly why spam prevention is difficult. Sorry for that.

[ Parent | on HN ]

Re:Ask HN: Let's build an HN uBlacklist to improve our Googl... by nixcraft on Wednesday January 05, 2022 @03:19AM (#29806477)

Yes, imagine if this blocklist becomes mainstream and used by major other adblockers or extensions? Then, there is no central place where one man's project can ask to remove my domain, and it will vanish. I often read on HN how much the web is centralized, and then we come across resources that kill independent blog/sites because of an error on the list maintainers part.

[ Parent | on HN ]

Re:Ask HN: Let's build an HN uBlacklist to improve our Googl... by PinkSheep on Thursday January 06, 2022 @01:49AM (#29820245)

As a user I think I've put your website on my mental "avoid it" list for its design. I've opened a page now and I feel like I'm instantly in a tunnel vision mode. For UX: it's not a pleasure to scroll up & down; maybe there's also a psychological element about the main content area being so slim in width.

The other comment made me remember there was captcha too, right? I had been using my own rented server as a VPN for all my internet access. But I'd have never blocked it for a public list - I've read the 'about me' page.

[ Parent | on HN ]

Re:Ask HN: Let's build an HN uBlacklist to improve our Googl... by ffhhj on Tuesday January 04, 2022 @03:17PM (#29799776)

> some of the top answers on StackOverflow were copied from my work without giving any credit to me

That's really frustrating. I'm building a faster search engine for programming queries and just added your site cyberciti.biz as a recommended and curated source of Unix/Linux material. Hope more devs get aware of your work and you (and your collaborators) receive the credits deserved. Thanks for your work of many years.

[ Parent | on HN ]

Re:Ask HN: Let's build an HN uBlacklist to improve our Googl... by nixcraft on Wednesday January 05, 2022 @03:20AM (#29806481)

Thank you. Do ping me when your work is ready. I will share it on Twitter :)

[ Parent | on HN ]

Re:Ask HN: Let's build an HN uBlacklist to improve our Googl... by nitrogen on Tuesday January 04, 2022 @12:45PM (#29797545)

What CDN do you use? I was immediately asked to solve a captcha from my phone.

[ Parent | on HN ]

Re:Ask HN: Let's build an HN uBlacklist to improve our Googl... by nixcraft on Tuesday January 04, 2022 @12:56PM (#29797678)

Cloudflare sometimes triggers those when they think IP reputation is not good. Typically happens for data centre IP ranges as WAF has an anti-bot feature. So I know it is a problem for some.

[ Parent | on HN ]

Re:Ask HN: Let's build an HN uBlacklist to improve our Googl... by nitrogen on Tuesday January 04, 2022 @01:29PM (#29798159)

Maybe it's the fact that I don't use one of the three major US ISPs. Hopefully CDNs get used to the idea that there can be more than one fiber provider.

[ Parent | on HN ]

Re:Ask HN: Let's build an HN uBlacklist to improve our Googl... by nixcraft on Tuesday January 04, 2022 @01:47PM (#29798408)

Would you mind sharing the Cloudflare ray id displayed at the bottom of the screen when you see a captcha? I can look into it, and maybe be I will able to fix it too. Reply here or email me at [email protected]. HTH.

[ Parent | on HN ]

Re:Ask HN: Let's build an HN uBlacklist to improve our Googl... by nitrogen on Tuesday January 04, 2022 @05:10PM (#29801304)

Not sure if you changed your Cloudflare settings, or if Cloudflare changed something, but I'm no longer getting the captcha, so that's good, but sadly I can't help debug the original issue.

[ Parent | on HN ]