I keep seeing this excuse but it doesn't explain why google doesn't downrank obvious bad actors that have been showing up at the top for years, or returning results that don't have the most important word in your search terms, or why they don't offer settings to permanently filter some sites. It's a problem with incentives, not SEO spam.

Most likely because the only reliable on-page indicator for detecting low quality content is the amount of ads/trackers on page which is probably the only thing that they can not directly punish a site for.

Indeed. Google used to have a mechanism to report pages - was annoying when they removed that. Even if it was just the illusion of control, it was nice to think they at least pretended to care.

They don't curate the search index by hand, and "obvious bad actors" aren't really that obvious from a "well-definable heuristic" point of view. It's obvious to a human, but it seems hard to identify via an algorithm

The SEO spam exists in part because Google does a really bad job at surfacing any site that isn’t content heavy. This leads to sites needing to churn out keyword rich fluff content in order to get any organic traffic.

I have first hand experience with this. My niche, hyper local online service (of which everyone loves) was completely undiscoverable through Google despite the niche having nearly zero relevant results of any kind, and zero competition of any kind. It seems Google didn’t like the concise, no BS one-pager for my service.

My options were to continue to spend on Facebook ads perpetually or to create SEO spam content. Within literally a month or so of generating fluff articles I was gaining a ton of organic traffic and could stop ad spending altogether.

A good example of larger company doing this is Digital Ocean. Fortunately they’ve been extremely ethical about it and the content is genuinely very good and helpful.

DO is the best example of online content marketing.

What is a hyper local online service, like services for someone you closely relate to geographically or socially?

Could you give an example?

By hyper local I mean something relevant to a small geographic region like a city or state.

To be clear, that aspect was mostly tangential to my point.

I think the point is Google is really bad at understanding web apps, online services, or really any site that isn’t information rich.

The majority of web apps for example only really need a landing page and a sign up form. But if that’s all you have Google’s algorithm is going to show zero interest. And it seems that’s the case even if you put in the work and optimize the content and provide plenty of meta data.

Imagine you own a pizza joint, but the only way Google shows your site to any of your potential customers is if you dedicate to publishing an article about pizza every month. That’s basically the boat a lot of us are in with web apps, and other online services.

> I think the point is Google is really bad at understanding web apps, online services, or really any site that isn’t information rich.

You keep using that phrase, but it doesn't select for information richness, it selects for verbiage. This is at best orthogonal and more usually opposed to information richness.

Verbiage is the result of bad copy writing and/or lazy keyword stuffing. I don’t think they select for or reward verbiage, but for sure they’re not doing enough to treat it as a negative signal.

Well not directly, but you're heavily penalized for not keyword stuffing and then passing it through an AI tool until it is 'simple to read' (ie. says what you're trying to say extremely badly five times with almost-correct words to avoid anpiece of jargon).

As a result even the content made with earnest intent to communicate has to read exactly like blogspam in order to rank.

The other side of the coin is that users are so used to the "free" model (read: ad model) that they refuse to pay for anything directly.

Look at youtube. They show ads and there is a non-stop stream of complaints about how long/bad/annoying they are. They offer a paid subscription to remove all ads and people laugh at the audacity youtube has to ask for money.

Users DO NOT want to see/watch ads. Users DO NOT want to pay a subscription for "free" content.

Just look at the story of Vid.me, a youtube competitor start-up that promised ad-free and sub-free videos. It was a huge hit until it hit bankruptcy.

Actually, things are quite the opposite. 50 million people pay for YouTube Premium. 50 million people do want to pay for ad-free content.

That is only 1% of all Google users maybe, but it is the seed we need.

https://www.theverge.com/2021/9/2/22654318/youtube-50-millio...

I wouldn't be so sure, agree that there is an enormous challenge to convince people to pay for something they think they are getting for free, even when it's really not free ie paying for it with time, attention, privacy.

But create enough value, provide truly differentiated and unique experiences that ad-supported models can't, include privacy etc and there is a case to be made that over time a sizable enough segment will pay for that value that you can have a sustainable business.

That is certainly our bet at Neeva, and so far we have seen a lot of growth, interest and folks opting for the premium (paid version) over the basic. Even without a full push to encouraging the premium.

YT Premium is expensive. Much more expensive than users expect.

Most people underestimate the value of their attention. Youtube earned 7 billion USD in ad revenue last quarter.

The value of our attention is a shock to most people. I suspect that YouTube may actually be losing money (as compared to ad revenue), even at the $12 / month--a price many users find outrageous.

I am calling this the 'ad gap', which is difference between what an advertiser is willing to pay, and what the user is willing to pay to avoid the ad.

One way to look at it is that users are irrational. But it can also be seen as a testament to the effectiveness of advertising.

Exactly. YouTube Premium is loaded. Its pricing is based around selling YouTube Music and YouTube Originals. I don't want either. I'd happily pay 5$/month for a YouTube Premium experience.

Sure, but my guess is that YouTube is earning more than that in advertising from you, even sans music and oroginals.

We’re used to free because google decided free was the answer and they had so much clout, no other answer could compete. Now that their time is ending there will be room for other answers.

No, free has been an accepted paradym for for a long, long time. Simply think of a hundred years of radio advertising, and just short of that for tv.

> It's not a Google problem

Oh it is definitelt a Google problem.

I was in mood yesterday and I tried Google versus Kagi and You Code. Results of N out of N queries in other search engines were miles better in other search engines than Google.

I have now switched completely to code.you.com and Kagi from Google.

They are ready.

Kagi is jaw-droppingly amazing. With Google search quality declining, Bing being on average 95% as good as Google, and DuckDuckGo being 90% as good, I was doubting there was any way that search engines would ever improve. Kagi has removed that doubt completely. I ask for stuff and it finds it. I couldn't believe there could be such a useful product.

Doesn’t DuckDuckGo use bing for the search backend?

Apparently it uses Bing plus some secret sauce although in my experience the results are a bit worse than pure Bing. Having said that I have not performed a rigorous study, this is just my guess from using all these search engines over a prolonged period.

It was mostly Bing and Yandex. The latter was responsible for the odd Russian language results you'd get at times but presumably produced English results as well.

Bing may be that good for some, but every time I use it it stinks and I end up back on Google. Same for DDG. Then again, my queries tend to be pretty specific and are probably not a typical use case.

I really expected code.you.com to be tailored to coding, but the results are nonsensical. The $ got stripped, as well as any other characters like %, #, or _. Casing is also ignored.

Searching for $PATH results in this. Yes, I definitely wanted these results:

> PATH (People Acting To Help), Inc. is a comprehensive Behavioral / Mental Health and Intellectual Disabilities center located in Northeast Philadelphia.

> Better health moves humanity forward | PATH The scope of our work is vast because it must be. Billions of people are still underserved or marginalized by inequitable systems. To close the gaps, we advance progress in dozens of health areas, from epidemic preparedness, HIV/AIDS, and malaria, to maternal and newborn care, sexual and reproductive health, tuberculosis, and more.

https://imgur.com/a/rGxZgrG

I'm a product manager working on code.you.com. This is good feedback and an interesting suggestion. We'll work on adding those special characters, variables and casing to make the search better for coding.

We're just getting started but working really hard to make this something special. There is a fairly active slack community if you have any other suggestions and want to share. thx.

Thanks for being engaged with your users. It's a rare thing.

Good luck with the product! I'd also suggest giving priority to sites which use <pre> or <code> blocks in their HTML.

Incredibly dumb and naive to assume that if Kagi becomes even remotely successful, the SEO bandwagon will not target them.

Kinda OT, but goddamn is it ballsy of Kagi to make their logo a G.

Come on'! Google owns pretty much everything on the web, we can't let them own the letter g too ;)

Never heard about those two.

I don't know if I can agree, yes SEO spam is probably worse but it's grown side-by-side with Google, both sides have more money and resources.

SEO spammers decided to improve their tools/techniques. Google realized they could earn just as much by replacing their grep-for-the-web search-engine with some NLP, ML and fuzzy matching on what I strongly suspect to be a tiny subset of the index they had before.

I understand the reasoning and I admit it does a really good job at correcting mistakes but I often wonder if anyone working on the search-engine actually uses it to find development related information.

Maybe I'm in some kind of A-B testing hell but my results often show 1 relevant random url and a tiny a stackoverflow box with other relevant information and then always the same 10-20 stackoverflow/github copycats. Scrolling down just loops those same sites over and over again. Similarly I've never had Google link to the official python docs, it's always some spammy website that copy pasted a docstring and decorated it with ads/share buttons.

I don't understand how that's not bothering people that actually work on the search-engine.

You right, but Google at least can try to fix it. For example: allow users to hide sites from search results, allow users to downrank sites in search results, etc. etc.

Then it's going to be gamed, again. This time by businesses who would love to drown their competitors search results, or malicious players who want to attack an entity's web presence.

It's much harder to game if the rules are different for each consumer (and yeah, most people won't bother, but then they can't really complain either)

They just need a hidden karma system that ignores the contributions from bad actors.

I understand that to some degree this is an arms race between business SEO optimization and Google's SEO ranking algorithms but I don't understand why Google upranks websites with obvious annoyances like the 3 popups in the case of OP's website, excessive amount of adtracking, and unreasonably large website sizes. In those cases, I can only suspect Google upranks them because they provide ad revenue to Google.

It is a google problem as well as a web problem.