Google is intentionally dropping five year old results from the index and prioritizing five hour old results. They have complete control over whether they're doing this. Part of SEO is making five year old results look like five hour old results, which is something that Google did to spammers, not something that spammers did to google.
Google has decided that it is more profitable to make sure that somebody who types "chicken" into the search box gets either chicken restaurants nearby, or listicles and video covering the story on the news today about a chicken that scared away a bear. They've given up on depth.
I mean, forgive me for thinking that "chicken" is actually a term referring to live birds. Instead, humanity's view is clearly very distorted by our stomachs.
One does not use the internet for facts. One uses the internet to be directed towards where to spend their money and/or time as sold to the highest bidder, who is just looking to line their pockets with money--a digital sugar rush for digital stomachs. The internet doesn't think with its brain, it doesn't tell you what you want to know, it thinks with its stomach and expects you to as well.
edit: and to directly address what you said, excusing a search engine's loss of content because sites have disappeared because the search engine dropped them is circular.
After testing with marginalia I found that much of it is still there - it is just suppressed by Google and Bing.
Google's "problem" is that they need to cater to the most common intents per query (see: lowest common denominator), and believe it or not, those intents are usually not biased toward in-depth educational content - isn't this why HN is such a nice niche?
Google could think about adding a different search result sorting option that attempts to prioritize high quality, thought provoking, or otherwise educational content above all else, but it makes sense to me why their primary search ranking algorithm works the way it does.
These days, the search volume went way up and my article went down to 6-10. The first three spots merely give a rough explanation of my in-depth topic. Quite insulting actually.
But upon further reflection I realized that most people these days only want a rough definition and no in-depth explanation. Google gives this to them.
The people who used to search for my keyword now add 2-3 more words to get to my in-depth article. The typical long tail.
What I find worrisome is that sometimes, the long tail query I use to find quality content gives me the same results as the unspecific short query. Some stuff is close to impossible to find without knowing a very topic-specific thing.
Preheat oven, insert chicken.
A bad search deserves a bad result. If you walked up to me and just said one word "chicken", I'd be insulted. If you said one word "chicken" while pointing at your mouth, I'd perhaps think you spoke little English and wanted a place to eat chicken. Providing a bit more info is useful to get decent results.
As Jerry Maguire once said, "help me help you"
Try something like this (just imagine a word or two that would appear in quality content about chicken and put it into marginalia):
https://search.marginalia.nu/search?query=poultry+farming&pr...
> We might be starting from the (slightly) false premise that the internet is full of quality content.
Well maybe it isn't but at least using Marginalia I get more of the results I want for these kinds of searches.
(I've also had good luck with a query about git a few weeks ago and yesterday with one about kubernetes.)
What knowledge cant be found with appending "wiki" to any keyword, what person cant you find with linkedin, what movie cant you find on a private bt tracker, what news cant you have from reddit ?
I think people over estimate what the web was before Google and under estimate how much easier it s been to find anything.
Name me one thing you struggled to find on google and Ill probably be able to describe you the nightmare it would have been 20 years ago
Edit: This is a better example: https://www.mechtype.com/category/reviews/
I'll show you what I'm looking for:
https://labs.tomasino.org/mechanical-keyboards/
https://www.paritybit.ca/blog/vortex-vibe-keyboard
https://wiki.installgentoo.com/wiki/Mechanical_keyboards
https://www.vegard.net/the-ultimate-keyboard-part-2-switches...
http://boxbase.org/entries/2015/nov/30/ergodox/
Where are those types of search results?
This is probably a better example: https://www.mechtype.com/xiaomi-yuemi-pro-mk02-review/#more-...
There are some basic economic realities underpinning why most people take the time to create content like this. I don't think it's reasonable to assume that just because someone expects to make a small amount of money from creating a review like that (with either affiliate links or buy links), that that person's opinion is inherently at conflict with honesty just because there's a monetary element. It may be true for many sites and many people, but it's pretty straightforward to parse whether someone is writing a review earnestly and passionately rather than for $.
Given Sturgeon's law (90% of everything is shit), why are most of the reviews 4/5 or above?
> There are some basic economic realties underpinning why people take the time to create content like this. I don't think it's reasonable to assume that just because someone expects to make a small amount of money from creating a review like that, that that person's opinion is inherently at conflict with honesty just because there's a monetary element.
I updated the post with several examples of the type of content I want you to find.
> This is probably a better example: https://www.mechtype.com/[...]
This one has affiliate links
The point I'm making is that I think your underlying premise, that all sites which have any kind of monetary element to them are fundamentally corrupt in terms of intention or conflict of intention, is flawed. I've now read multiple reviews on https://www.mechtype.com/category/reviews/ that are quite clearly the work of a keen mechanical keyboard hobbyist with sufficient details, knowledge and effort to lead me to believe the work is genuine and useful rather than merely financially motivated alone or spammy. There are many examples of what you suggest, and I agree that many sites with affiliate links and ad popups are disingenuous and fall much closer to the spam end of the spectrum. But there are still useful results if the user is remotely competent at parsing information. If your argument is that users shouldn't need that competence, and search engines should be kinder to less competent users in that regard, then that's a separate (and probably more interesting) conversation.
That site also has a clear affiliate disclosure along with this:
https://www.mechtype.com/disclosures/
>Paid Reviews/Sponsored Posts. MechType does NOT accept paid reviews or sponsored posts. All thoughts expressed within our reviews are our own opinions. However, MechType may on occasion accept product samples for review. When a product sample is provided for review it will be clearly and fully disclosed in the post along with the company/persons who provided the sample.
I did not in fact claim that. You are confusing me with someone else in this thread. I agree that some things are hard to find on google, but I reject the notion that anything with a financial element is fundamentally corrupt in terms of intention.
One incentive would be that they earn more money through affiliate (or the company is paying them) to promote one product over the other.
Ad less pages. That provide the content without trying to monetize it.
> The problem is that search engine results – especially on mobile – are not how you find that any longer.
I think the biggest story of the web in the last decade is the slow but steady decline of Google.
Google has fallen victim to spam. Goodhart's Law wins in the end. Almost every site on Google's front page is there because people have spent a lot of time and money optimizing it for search engine ranking. But just because a page is good at gaming Google's algorithm does not mean it is the most useful anymore. Google put up a good fight, but it seems like they have finally lost.
It also seems like the company has lost it's way. I was recently using Yandex mobile browser to browse some Russian sites and it can translate text inside images and now even voice inside videos. Chrome on the other hand often fails to detect other languages and therefore does not translate even text. It feels like even 15 years ago (when I was becoming a Google fanboy, I have since grown out of it) I would have expected Google to get there before some obscure Russian company. I get it, Russian web is smaller than English web and therefore it is probably more useful for Yandex, but Google's revenues are many times that of Yandex, they should ideally be getting there by now.