To use a specific example, the other day I was trying to learn more about programming shaders. There is an unending number of basic level tutorials: what is a shader? There are probably 500,000 people searching for that topic. But if I want to figure out a UV scrolling texture, I'm screwed, there's almost nothing. Because the audience for that is maybe 500 people.
There's just no financial (or even reputational!) incentive for anyone to write an article / make a video on something requiring deeper expertise. There is stuff out there, of course, created by the tiny minority of passionate enthusiasts or dedicated professionals. Google is really searching for a needle in a very, very large haystack. And I think Google (and other search) has been more and more challenged by that over the years as the internet moved from a niche enthusiast place to a mass market place. Particularly when all the hay is spending massive amounts of effort on trying to pretend to be a needle.
tl;dr I despise using Google (or DuckDuckGo) these days but I don't think all the blame should be assigned to the engines, they're actually facing a tough problem. (Not that you personally were saying that, but this is mainly a thread of hating on Google.)
See I think this becomes chicken/egg real quick too, though I do think Google is more to blame. Content creators are responding to the incentives google created. Think back to when sites would have all sorts of keywords buried in metadata and/or have blogs that were complete nonsense full of keywords that weren't easily findable on their sites. They did it because they were rewarded. So what did google do? They penalized it, and voila! It stopped. This was like...2014? Maybe earlier?
Ultimately if google wants better search results they need to go back to the drawing board. They need to disincentivize blog spam.
Sometimes I've fantasized about Google having human editorial that would evaluate these content farms as they grow and make the nuanced decisions of which to derank. But then, if they really did that, they'd get sued or regulated pretty quickly.
I feel weird arguing for Google's side since I actually do have a lot of criticisms and generally dislike Google search. I just think this content farm issue looks very thorny; humans are built to deceive each other and at some point of sophistication that gets very difficult to control.
I’m not saying I have all the answers here but I feel like this shouldn’t be too hard to at least make a dent in.
If I have to sift through one more useless AI-generated/copy-paste article where they spend half the damn thing explaining the most basic stuff and then still not answering my question (or giving a simple "yes/no"), I'm going to lose it. It's not even just technical answers. I have to basically type "...reddit" or some forum I like to get ANYTHING useful anymore. It's even just basic media questions.
Example: "Is there an [IP sequel] being made?"
First 20 results: "Is there an [IP sequel] being made? Read more to find out. First what is IP? IP was released in 2012. It was made by such and such starring such and such as they go on an adventure to do the thing..." 4 paragraphs later "There is no information on a sequel. Thanks for reading!"
Yes I've learned to spot these as have most folks around here I imagine, but holy hell they will eat 2-3 pages of results sometimes! It's infuriating!