This better echoes my personal experience with the decline of Google search than TFA: it seems to be connected to the increasing use of ML in that the more of it Google put in, the worse the results I got were.
The people I see who are most excited about ML are business types who just see it as a black boxes that makes stock valuation go vroom.
The people that deeply love building things, really enjoy the process of making itself, are profoundly sceptical.
I look at generative AI as sort of like an army of free interns. If your idea of a fun way to make a thing is to dictate orders to a horde of well-meaning but untrained highly-caffienated interns, then using generative AI to make your thing is probably thrilling. You get to feel like an executive producer who can make a lot of stuff happen by simply prompting someone/something to do your bidding.
But if you actually care about the grit and texture of actual creation, then that workflow isn't exactly appealing.
Using English, instead of C, to get a computer to do something doesn't turn you into a beaurocrat any more than using Python or Javascript instead does.
Only a person that truly loves building things, far deeper than you'll ever know, someone that's never programmed in a compiled language, would get that.
If one uses English in as precise a way as one crafts code, sure.
Most people do not (cannot?) use English that precisely.
There's little technical difference between using English and using code to create...
... but there is a huge difference on the other side of the keyboard, as lots of people know English, including people who aren't used to fully thinking through a problem and tackling all the corner cases.
No one can, which is why any place human interaction needs anything anywhere close to the determinancy of code, normal natural langauge is abandoned for domain-specific constructed languages built from pieces of natural language with meanings crafted especially for the particular domain as the interface language between the people (and often formalized domain-specific human-to-human communication protocols with specs as detailed as you’d see from the IETF.)
Still needed domain experts back then, and, IMHO, in years/decades to come
really, my impression is the opposite. They are driven by doing cool tech things and building fresh product, while getting rid of "antiquated, old" product. Very little thought given to the long term impact of their work. Criticism of the use cases are often hand waved away because you are messing with their bread and butter.
I think we also need to be aware that this business layer above us that often sees __computers__ as a magic box where they type in. There's definitely a large spectrum of how magical this seems to that layer, but the issue remains that there are subtleties that are often important but difficult to explain without detailed technical knowledge. I think there's a lot of good ML can do (being a ML researcher myself), but I often find it ham-fisted into projects simply to say that the project has ML. I think the clearest flag to any engineer that this layer above them has limited domain knowledge is by looking at how much importance they place on KPIs/metrics. Are they targets or are they guides? Because I can assure you, all metrics are flawed -- but some metrics are less flawed than others (and benchmark hacking is unfortunately the norm in ML research[0]).
[0] There's just too much happening so fast and too many papers to reasonably review in a timely manner. It's a competitive environment, where gatekeepers are competitors, and where everyone is absolutely crunched for time and pressured to feel like they need to move even faster. You bet reviews get lazy. The problems aren't "posting preprints on twitter" or "LLMs giving summaries", it's that the traditional peer review system (especially in conference settings) poorly scales and is significantly affected by hype. Unfortunately I think this ends up railroading us in research directions and makes it significantly challenging for graduate students to publish without being connected to big labs (aka, requiring big compute) (tuning is another common way to escape compute constraints, but that falls under "railroading"). There's still some pretty big and fundamental questions that need to be chipped away at but are difficult to publish given the environment. /rant
The key difference is how tolerant the specific use case is of a probably-correct answer.
The things recent-AI excels at now (generative, translation, etc.) are very tolerant of "usually correct." If a model can do more, and is right most of the time, then it's more valuable.
There are many other types of use cases, though.
On the other hand ML has absolutely revolutionised translation (of longer text), where having a model containing prior knowledge about the world is essential.
I am probably the 0.01% of Tesla drivers who have the computer chime when I exceed the speed limit by some offset. Very regularly, even when FSD is in “chill” mode, the model will speed by +7-9 mph on most roads. (I gotta think that the young 20 somethings who make up Tesla's audience also contributed their poor driving habits to Tesla's training data set) This results in constant beeps, even as the FSD software violates my own criteria for speed warning.
So somehow the FSD feature becomes "more capable" while becoming much less legible to the human controller. I think this is a bad thing generally but it seems to be the fad today.
Here you can see it detected an obstacle (as evidenced by info on screen), made a decision to stop, however it failed to detect existence of the object right in front of the car, promptly forgot about the object and decision to stop and happily accelerated over the obstacle. When tackling a more complex intersection it can happily change its mind with regards to exit lane multiple times, e.g. it will plan to exit on one side of a divider, replan to exit onto upcoming traffic, replan again.
They are lying with statistics, for the more challenging locations and conditions the AI will give up and let the human take over or the human notices something bad and takes over. So Tesla miles are miles are cherry picked and their data is not open so a third party can make real statistics and compare apples to apples.
*Disclaimer: as someone who's not an AI researcher but did quite some human translation works before.
Argh. My PTSD from writing ONVIF drivers just kicked in.
and how difficult it is to program those GPU to do ML
ML isn't like that. It's new. It's different. It may not succeed in the ways we expect; it may even look dumb in hindsight. But it absolutely represents a genuinely new paradigm for computing and is worth studying and understanding on that basis. We look back to SOAP and see something that might as well be forgotten. We'll never look back to the dawn of AI and forget what it was about.
[1] For anyone who missed that particular long-sunken boat, SOAP was a RPC protocol like any other. Yes, that's really all it was. It did nothing special, or well, or that you couldn't do via trivially accessible alternative means. All it had was the right adjective ("XML" in this case) for the moment. It's otherwise forgettable, and forgotten.
Anyone claiming it’s some sort of snake oil shouldn’t be taken seriously. Certainly the current hype around it has given rise to many inappropriate applications, but it’s a wildly successful and ubiquitous technology class that has no replacement.
Reading these comments I thought I stepped into some alternate timeline when we don't already have widespread ML all over the place.
Like, nobody does rules-based image recognition for a decade now already!
This new ML that's supposed to be the basis for an entire new economic wave, that I mostly dislike.
But I guess that's how we build new things... We explore and throw away 80% of what we've built.
Chatgpt generated the entirety of the above w/ me tweaking one line of code and putting creds in. I could have written all of the above, but it probably would have taken 20-30 minutes. With chatgpt I banged it out in under a minute, helped a colleague out, and went on my way.
Chatgpt absolutely is a real advancement. Before they released gpt4, there was no tech in the world that could do what it did.
They routinely give me brain-dead suggestions such as to watch a video I just watched today or yesterday, among other absurdities.
These days my biggest gripe is that they put unrelated ragebait or clickbait videos in search results that I very clearly did not search for - often about American politics.
Now the "related" section is gone in favor of "recommended" samey clickbait garbage. The relations between human interests are too esoteric for current ML classifiers to understand. The old Markov-chain style works with the human, and lets them recognize what kind of space they've gotten themselves into, and make intelligent decisions, which ultimately benefit the system.
If you judge the system by the presence of negative outliers, rather than positive, then I can understand seeing no difference.
I would go further and say that it is impossible. Human interests are contextual and change over time, sometimes in the span of minutes.
Imagine that all the videos on the internet would be on one big video website. You would watch car videos, movie trailers, listen to music, and watch porn in one place. Could the algorithm correctly predict when you're in the mood for porn and when you aren't? No, it couldn't.
The website might know what kind of cars, what kind of music, and what kind of porn you like, but it wouldn't be able to tell which of these categories you would currently be interested in.
I think current YouTube (and other recommendation-heavy services) have this problem. Sometimes I want to watch videos about programming, but sometimes I don't. But the algorithm doesn't know that. It can't know that without being able to track me outside of the website.
* They could let me directly enter my interests instead of guessing
* They could classify videos by expertise (tags or ML) and stop recommending beginner videos to someone who expresses an interest in expert videos.
* They could let me opt out of recommending videos I've already watched
* They could separate sites into larger categories and stop recommending things not in that category. For me personally, when I got to youtube.com I don't want music but 30-70% of the recommendations are for music. If the split into 2 categories (videos.youtube.com - no music) and (music.youtube.com - only music) they'd end up recommending far more to me that I'm actually interested in at the time. They could add other broad categories like (gaming.youtube.com, documentaries.youtube.com, science.youtube.com, cooking.youtube.com, ...., as deep as they want). Classifying a video could be ML or creator decided. If you're only allowed one category they would be incentive to not mis-classify. If they need more incentive they could dis-recommend your videos if you mis-classify too many/too often).
* They could let me mark videos as watched and actually track that the same as read/unread email. As it is, if you click "not interested -> already watched" they don't mark the video as visibly watched (the red bar under the video). Further, if you start watching again you lose the red-bar (it gets reset to your current position). I get that tracking where you are in a video is something that's different for email vs video but at the same time (1) if I made it to 90% of the way through then for me at least, that's "watched" - same as "read" for email and I'd like it "archived" (don't recommend this to me again) even if I start watching it again (same as reading an email marked as "read)
>let me directly enter my interests
Theres a general problem in the tech world where people seem to inexplicably disregard the issue of non-reducibility. The point about the algorithm lacking access to necessary external information is good.
A dictionary app obviously can't predict what word I want to look up without simulating my mind-state. A set of probabilistic state transitions is at least a tangible shadow of typical human mind-states who make those transitions.
Someone probably changed the engine that shows videos for you - exactly as with search.
Or when they would show more than 3 results before spamming irrelevant videos.
Or when they didn't show 3 unskippable ads in a 5 minute video.
Or when they had a dislike button so you would know to avoid wasting time on low quality videos.
Wait what?! You "Consume Content" on a COMPUTER? What are you some kinda grandpa? Why aren't you consuming content from your phone like everyone else? Or casting it from your phone to your SMART TV! Great way to CONSUME CONTENT!
CONSUME CONTENT CONSUME CONTENT CONSUME CONTENT CONSUME CONTENT CONSUME CONTENT CONSUME CONTENT CONSUME CONTENT CONSUME CONTENT CONSUME CONTENT
I don't know about the TV though.
> Or when they didn't show 3 unskippable ads in a 5 minute video.
I also use Firefox for Android, which has Addon support. Ublock Origin works on the phone and disables a a lot of the ad horror.
It feels a bit funny asking this, since we're talking about Google (i.e. YouTube), but did you mean ;) PipeTube? I know there is a PeerTube too.
If I entirely avoid watching any popular videos, the recommendations are quite good and don’t seem to include anything like what you are seeing. If I don’t entirely avoid them, then I do get what you are seeing (among other nonsense).
Honestly, I'd prefer my voice assistant (siri mostly) to be like that as well. It was at first, and I think everyone hated that lol.
But I associate YouTube promotions with garbage any how. The few things I might buy like Tide laundry detergent are entirely despite occasional YouTube promotion.
Besides the religious crap, ill randomly get shit in India in hindu, having had not watched anything Indian and not even remotely Indian.
Hindi is the word for the language, bro.
From a user perspective, Shorts highlights a specific format of YouTube that happened to have been around for a lot longer than people realize. TikTok isn't anything new, Vine was doing exactly the same thing TikTok was a decade prior. It was shut down for what I can only assume was really dumb reasons. A lot of Viners moved to YouTube, but they had to change their creative process to fit what the YouTube algorithm valued at the time: longer videos.
Pre-Shorts, there really wasn't a good place on YouTube for short videos. Animators were getting screwed by the algorithm because you really can't do daily uploads of animation[0] and whatever you upload is going to be a few minutes max. A video essayist can rack up hundreds of thousands of hours of watch time while you get maybe a thousand.
(Fun fact: YouTube Shorts status was applied retroactively to old short videos, so there's actually Shorts that are decades old. AFAIK, some of the Petscop creator's old videos are Shorts now.)
But that's why users or creators would want to use Shorts. A lot of the UX problems with Shorts boils down to YouTube building TikTok inside of YouTube out of sheer corporate envy. To be clear, they could have used the existing player and added short-video features on top (e.g. swipe-to-skip). In fact, any Short can be opened in the standard player by just changing the URL! There's literally no difference other than a worse UI because SOMEONE wanted "launched a new YouTube vertical" on their promo packet!
FWIW the Shorts player is gradually getting its missing features back but it's still got several pain points for me. One in particular that I think exemplifies Shorts: if I watch Shorts on a portrait 1080p monitor - i.e. the perfect thing to watch vertical video on - you can't see comments. When you open the comments drawer it doesn't move over enough and the comments get cut off. The desktop experience is also really bad; occasionally scrolling just stops working, or it skips two videos per mousewheel event, or one video will just never play no matter how much I scroll back and forth.
[0] Vtubers don't count
There's videos I'll watch multiple times, music videos are the obvious kind, but for some others I'm just not watching/understanding it the first time and will go back and rewatch later.
But I guess youtube has no way to understand which one I'll rewatch and which other I don't want to see ever again, and if my behavior is used as training data for the other users like you, they're probably screwed.
I got nothing against ML in principle, but if the model doesn't do the right thing then you can just end up stuck. Also, it often burns a lot of resources to learn something that was obvious to human domain experts anyway. Plus the understandability issues.
The pathologies of big companies that fail to break themselves up into smaller non-siloed entities like Virgin Group does. Maintaining the successful growing startup ways and fighting against politics, bureaucracy, fiefdoms, and burgeoning codebases is difficult but is a better way than chasing short-term profits, massive codebases, institutional inertia, dealing with corporate bullshit that gets in the way of the customer experience and pushes out solid technical ICs and leaders.
I'm surprised there aren't more people on here who decide "F-it, MAANG megacorps are too risky and backwards not representative of their roots" and form worker-owned co-ops to do what MAANGs are doing, only better, and with long-term business sustainability, long tenure, employee perks like the startup days, and positive civil culture as their central mission.
An example of this I see is how new leaders come in and hit hard to cut costs. But the previous leader did this (and the one before them) so the system/group/company is fairly lean already. So to get anywhere near similar reductions or cost savings it typically means cutting more than fat. Which it's clear that many big corps are not running with enough fat in the first place (you want some fat! You just don't want to be obese!). This seems to create a pattern that ends up being indistinguishable from "That worked! Let's not do that anymore."
Example: User Growth & Customer Engagement
Have to have user growth and retention. If you looked at just one or the other, you'd be missing half the equation.
1) People setting the metrics 2) People implementing/calculating the metrics 3) People working on improving the metrics (ie product work)
2 is specially complicated for a lot of software products because it can some times be really hard to measure and can be tweaked/manipulated. For example, the MAU twitter figures from the buyout that Musk keeps complaining about, or Blizzard constantly switching their MAU definition.
Often 2 and 3 are the same people and 1 is almost always upper management. I argue that 1 and 2 should be a single group of people (that doesn't work on the product at all) and not directly subject to upper management and not tracked by the same metrics they implement (or tracked by any metrics at all).
I would love to join a co-op producing real human survival values in an open source way. Where would you suggest that I look for leads on that kind of organization?
While DDG, Brave, Kagi etc are working generously to replace Google search. The other areas that I think get less attention and needs to be targeted to successfully dismantle them and their predatory practices are Google maps and Google docs.
Maps are hard because it requires a lot of resources and money and whatever but replacing docs should be relatively easier.
There is big tech open source consortium working on maps now to commoditize it: https://siliconangle.com/2022/12/15/aws-microsoft-meta-tomto...
Not sure it'll work. I think half the advantage comes from the integration across all these tools (maps, search, etc). Have you ever tried to use duckduckgo? It surprised me what I take for granted in Google's user experience.
FWIW, Kagi is built on top of Google search, so yes it's "replacing" (for you and me) a dependence on Google search, but it is categorically not a from-the-ground-up replacement for Google search.
It can't have ads, and it can't hide any knowledge that exists which could help the user.. even if the knowledge is proprietary.
It must repeal copyright laws by force. It must drain all silos and know all things. And it must utilize the entirety of the library Genesis.
So... people go walk up for handouts from VCs....and the story begins lol.
Co-ops could go the angel/VC route for funding if they don’t give up a controlling share.
if they were unable to do some AB testing between a ML search and a non-ML search, they deserve their failure 100%
there are not enough engineers blowing the whistle against ML
It would have been a worse outcome for Google if they had stuck to their no ML stance and then had Bing take over search because they were a generation behind in technology.
Unfortunately, Google evaluates employees by the complexity of their work. "Demonstrates complexity" is a checkbox on promo packets, from what I've heard.
Naturally, every engineer will try to over-complicate things just so they can get the raises and promos. You get what you value.
Same is happening to YouTube as well. Feels like it’s nothing but promoters pushing content to gain followers to sell ads or other stuff because nobody else’s videos ever surface. Just a million people gaming the algorithm and the only winners are the people who devote the most time to it. And by the way, would I like to sign up for their patreon and maybe one of their online courses?
0: for context, us doj does not take antitrust action against companies simply for market dominance; it requires market dominance plus an anticompetitive action. However, they don’t like monopolies, so effectively any pretext can be used — see the apple lawsuit or the 90s ms lawsuits for how little it takes.
https://en.m.wikipedia.org/wiki/Antitrust_cases_against_Goog...
To summarize it: Google reverted an algorithm that detected SEO spams in 2019.
(Note that I never work for Google and I don't know whether it's true or not. It's just what this article says.)
This isn't about what is possible.
It is about Google not wanting to say goodbye to the sweet dollars from spammy sites.
Otherwise making the probably number one requested feature, a personal block list, wouldn't have been impossible for a company with so many bright minds.
I mean: little bootstrapped Kagi had it either from the beginning or at least since shortly after they launched.
People always think they lost against SEO spam. But my main reason for quitting as soon as an alternative showed up was because they started to overrule my searches and search for what they thought I wanted to search for.
For a while I kept it at bay by using doublequotes and verbatim but none of those have worked reliably for a decade now.
That isn't SEO spam. That is poor engineering or "we know better than you" attitude.
The results suggest that Quebec does not have an NHL team, because it confuses the province of Quebec with Quebec City. Montreal, in Quebec, has the Montreal Canadiens and this isn't mentioned in the search results at all.
And who forgot the recent Reddit story.
Search for a while hasn’t been about searching the web as much as it has been about commerce. It taps commercial intent and serves ads. It is now an ad engine; no longer a search engine.
If you want to purchase consumer products at your own expense and offer an impartial opinion on each of them then you will have no problem getting ranked highly on google. You will lose a lot of money doing so, however, and will also be plagiarized to death in a month. The sites you want to be rid of will outrank you for your own content, I have been there and have the t-shirt.
Google doesn’t have to return the SEO-optimized page. Google has other options:
- Return 10 results of the 10 top products,
- Derank any site that seems SEO-optimized,
- Derank any commercial site,
- Derank any site with a cookie banner (implying the user is tracked and the writer is trying to write what the user wants to read) or the infamous mailing list popup,
- Prioritize comparisons from brick-and-mortar journals, or give credentials to other vectors of trust,
- Act as a paid directory, where only paid answers appear,
- Return individual positive and negative comments about products, extracted from review pages, maybe even in a graph (“Good for USB-C according to 95% of the reviews, provides an electric shock according to 7% of non-affiliated comments”).
There WERE many options. Google CHOSE to rank awful sites that provide decreased value, and worse than that, it chose that all other sites won’t be viable, killing them. Google chose the face of the internet today.
[1] - https://en.wikipedia.org/wiki/Breakup_of_the_Bell_System
And I know they could because someone did make that list and posted it here last year.
Not low quality pages that spammed high ranking words but pages that simply wasn't related to the query at all as evidenced by the fact that they didn't contain the keywords I searched for at all!
I wish there was some way to tell "please ignore all videos that contain these strings, and I don't mean only for next 2 weeks".
Youtube gets their ads revenue from before/during video, so they can be nicer to users.
I think it's more likely that Google stopped really caring.
There are various kinds of SEO - internal: technical, on-page and external. A long time ago Google had an epiphany that instead of trying to make sense out of sites themselves they could offload that effort to website administrators and started ranking pages how well they implement technical elements helping Google index the web. For a very long time that was synonymous with white-hat SEO. Since Google search was in part based on web-of-links, various shady tactics to inflate number of indexed backlinks and boost rankings. That was black-hat SEO.
These days Google search puts tremendous focus on on-page SEO. So much that as long as the internal structure of a site is indexable (no dead links, internal backlinks, meta info) it is typically better to hire copywriters spitting out LLM-like robotic mumblings than to try and optimize further.
When the 'trusted websites' caught on and embraced the game, Google was apparently helpless to stop it.
Its really easy to spot the crap websites that are scaping content-creating websites ... because they monetize by adding ads.
If Google was _only_ selling ads on the search results page, then it could promote websites that are sans ads.
Instead, it is incentivised to push users to websites that contain ads, because it also makes money there.
And that means scraping other sites to slap your ads onto them can be very profitable for the scammers.
This is like in that Steve Jobs video about product people being kicked out and exchanged by ones who dont care about product:
https://m.youtube.com/watch?v=P4VBqTViEx4
They will not make good search. That is not their priority.
Personally, I use DDG on the daily basis, and it's mostly ok, but very-very far from perfect. More so, at least once in several days I have to switch to Google, because it is seriously better at updating the index, and DDG often fails to find something on some obscure forum, even if I know it's there (because I was a part of discussion myself!) and try to assist it with finding it as much as I can. Also, Google is immensely better at knowing local shops and finding products.
Also, Google search, being bad as it is, it still the only thing I find usable on mobile. First off, it's faster, it is integrated nicely into Pixel UI, and it's somewhat good at all these "more than just a search" type of things, like converting a timezone for me, showing wikipedia summary, flight schedule, etc. Also, integration with Google Maps, working hours and venue locations, it is actually far more reliable than, say, Tripadvisor.
Still, I feel reluctant to vendor-locking myself into payed service unless it's actually far better than everything else and can replace DDG and Google completely.
?
Tangential, but this is precisely the "problem" with Google search. Whatever the internal decision-making process was, Google search at some point embraced race to the bottom incentivizing outspending others, either by paying for ads or showing ads. This race is ultimately won by content scrapers/generators slapping ads on top and businesses selling stuff.
Anecdotally, there is a pet supply store near me. It's nearly impossible to find on Google maps. If I zoom over the shopping mall this particular store does not appear, if I search for "pet store" it does not appear. Only if I do search for "petstore inc." it appears in results and map. So Google knows about the store, but actively tries to hide it, presumably because Google does not make money off it.
> I have to switch to Google, because it is seriously better at updating the index
On one hand yes, Google is in some cases really quick at updating the index with new entries. However, at the same time it is equally good at updating the index with removals making old content very hard to find.
It's not "that much" better for some definitions of "that much".
But they're working on making the best search engine for their customers, and it does have a lot of features for helping make your search better and less ad-driven.
I was trying to find the age of an obscure local lava flow. Google was useless for it. Kagi had it on the third hit. So sometimes it's brilliantly better.
But what I like the most is that their incentives are aligned with mine (because I'm paying them to be).
Google is going to maximize revenue which means making it as shitty as possible without you leaving. How many ads can I cram down their throats before they split? Kagi is also maximizing revenue, but they want to make it as great as possible so you don't leave.
Are the results worth it? It's up to you, really. Try it for free--if you don't miss it after you run out of free searches, then it's not for you.
A bit farther down the page is a 'best headphones for 2020' article.
And this is the example result set they push on the home page to a potential buyer.
You guys pay for this thing?
The fact that you tried to pick on 2 results of the page for such a generic keyword, show that it's miles ahead of mainstream search engines.
I tried that same search on Google, bing, brave, yandex, even yahoo and needless to say the results were pretty much all SEO spam from websites such as NYTimes (how tf is NYTimes an authoritive source on purchasing headphones?). Where in Kagi you got a wide range of helpful results from: reddit, Quora, amazon, walmart, etc. To be honest brave and yandex were the closest behind. As far as I know the only weak point in Kagi at the moment is doing more local-oriented search.
Regardless of the quality of results (which mind you, are already superior overall), it'd be still worth paying $10, if only to support/invest in its non-ads search model and help nurture it. It's a great effort for that alone. Combine both the model and high-quality results and it's the best in class.
I didn't 'try to pick on' - I pointed out two garbage results in a query that they literally push you to from the home page as examples for potential customers. If those results aren't doing what people claim (not highlighting seo spam) then I'm not really left with any faith that the queries they don't elevate to their home page will be better.
Acqui-hire. So what happened was in around 2010 or so a voice-over artist named Lauren Dragan who I think was already dabbling in professional tech journalism, wanted to write about headphones and microphones since she was getting really opinionated about them in her VO work.
So she contributed an article to “The Wirecutter,” which was trying to be like Tom’s and Engadget (I think they then dropped “the” from their name? Which makes one want to abbreviate as WC which is just tragic). I think it was just a freelance article on “audiophile headphones”...?
Well, the audiophile community online was growing etc. and this proved to be remarkably successful because it gave the audiophiles some professional validation, right? “I work in audio booths, I have to listen super closely, I know what I am talking about.” So it made money for The Wirecutter and they pitched her on “if we just bought you dozens of headphones online would you take notes and make a rec” and she's been doing stuff like that for them ever since.
Wirecutter broadened its focus to a lot of other topics, usually not with the same reliability—it really depends on the reviewer’s biases and such, and Lauren’s VO/audiophile bias of “I want my headphones to have a very flat EQ to match what's on the track, it's more important that they don't croak at higher volumes...” was something she could communicate well about in terms of sibilant highs or feeling too much or too little bass. Vs “we looked at air purifiers and, uh, they purify air!” ...
Meanwhile NYT was trying to grow their online presence as newspaper sales die... So they bought up Wirecutter, as a sort of “new journalism,” a “we wanted to get into this anyway, and it's easier if we don't try to build up the network effects ourselves but just take a site’s traffic who is already successful.” So yeah, they aqui-hired Wirecutter and put all their stuff on their domain and it kinda sucks now, but some of that were trends that were already beginning before they were acquired and there's still usually some decent data hiding in the “the competition” section of every “WC” article.
After that I only use search for technical problems, and mouth to mouth or specific authors for everything else.
Problem is that the rules of search engines created the dubious field of SEO in the first place. They are not entirely the innocent victim here.
Arcane and intransparent measures get you ahead. So arcane that you instantly see that it does not correspond with quality content at all, which evidently leads to a poor result.
I wish there was an option to hide every commercial news or entertainment outlet completely. Those are of course in on SEO for financial reaesons.
There's alway plugins or you can subscribe to Kagi, although I don't think there's any blocklist preconfigured for "all commercial news websites"
As for SEO spam a huge chunk of it would have disappeared I think if Google had created the much requested personal blacklist that we used to ask them for.
It was always "actually much harder than anyone of you who don't work here can imagine for reasons we cannot tell or you cannot understand" or something like that problem, but bootstraped Kagi managed to do it - and their results are so much better that I don't usually need it.
In a bid for attention, only the fraudsters are winning, well, the platforms are winning lots of money from selling advertising, I guess that's why they're perfectly fine with not fixing results and ranking for many years now. I'm not sure there is a way back to real relevance now, there's no incentive for these large companies to fix things, and the public has already become used to the gamified system to go back to behaving themselves.
My priors before reading this article were that an uncritical over-reliance on ML was responsible for the enshittification of Google search (and Google as a whole). Google seemed to give ML models carte blanche, rather than using the 80-20 rule to handle the boring common cases, while leaving the hard stuff to the humans.
I now think it's possible both explanations are true. After all, what better way to mask a product's descent into garbage than more and more of the core algorithm being a black box? Managers can easily take credit for its successes and blame the opacity for failures. After all, the "code yellow" was called in the first place because search growth was apparently stagnant. Why was that? We're the analysts manufacturing a crisis, or has search already declined to some extent?
There's a lot of ML hate here, and I simply don't see the alternative.
To rank documents, you need to score them. Google uses hundreds of scoring factors (I've seen the number 200 thrown about, but it doesn't really matter if it's 5 or 1000.) The point is you need to sum these weights up into a single number to find out if a result should be above or below another result.
So, if:
- document A is 2Kb long, has 14 misspellings, matches 2 of your keywords exactly, matches a synonym of another of your keywords, and was published 18 months ago, and
- document B is 3Kb long, has 7 misspellings, matches 1 of your keywords exactly, matches two more keywords by synonym, and was published 5 months ago
Yes, this is a statistical method. Guess what machine learning is and what it actually excels?
Ranking means deciding which document (A or B) is better to return to the user when queried.
Not writing a traditional forward-algorithm to rank these documents implies one of the following:
- You write a "backward" algorithm (ML, regression, statistics, whatever you want to call it).
- You don't use algorithms to solve it. An army of humans chooses the rankings in real time.
- You don't rank documents at all.
> Counting how many links are pointing to each document is sufficient if you know how long that link existed
- Link-counting (e.g. PageRank) is query-independent evidence. If that's sufficient for you, you'll always return the same set of documents to each user, regardless of what they typed into the search box.
At best you've just added two more ranking factors to the mix:
- document A
qie:
length: 2Kb
misspellings: 14
age: 18 months
+ in-links: 4
+ in-link-spamminess: 2.31E4
qde:
matches 2 of your keywords exactly
matches a synonym of another of your keywords
- document B
qie:
length: 3Kb
misspellings: 7
age: 5 months
+ in-links: 2
+ in-link-spamminess: 2.54E3
qde:
matches 1 of your keywords exactly
matches 2 keywords by synonym
- Which document matches your query better, A or B?
- How did you decide that, such that not only can you program a non-ML algorithm to perform the scoring, but you're certain enough of your decision that you can fix the algorithm when it disagrees with you ( >> debuggable and understandable by human search engineers )
Pagerank based ranking will not return the same set of pages. Its true that the ranking is global in vanilla version of Pagerank, but what gets returned in rank order is the set of qualifying pages. The set of qualifying pages are very much query sensitive. Pagerank also depends on a seed set of initial pages, these may also be set on a query dependent way.
All this is a little moot now, because Pagerank even defined in this way stopped being useful a long time ago.
There's got to be a better way.
Point being, there's a lot of amazing stuff that folks on the outside never would have seen, and it would be a shame for beancounters to ruin it all with decisions actively not "respecting the user".
The problem though, is the older systems had obvious problems, while the newer systems have hidden bugs and conceptual issues which often don't show up in the metrics, and which compound over time as more complexity is layered on. For example: I found an off by 1 error deep in a formula from an old launch that has been reordering top results for 15% of queries since 2015. I handed it off when I left but have no idea whether anyone actually fixed it or not.
I wrote up all of the search bugs I was aware of in an internal document called "second page navboost", so if anyone working on search at Google reads this and needs a launch go check it out.