Internal Search owners can push for better algos, but what if the algo causes revenue to fall? Are there strong forces strong enough within the organisation to ensure that search quality prevails?
If this is the case, the problem is existential. It can only be arrested at the very top
This gets close to the real root of the issue -- attention is monetizable independently of the quality of content. There would be much less incentive to create SEO spam if search engines negatively weighted pages with ads and affiliate links, and if manufacturers were barred (e.g. by the FTC) from owning or imitating reviewers.
This is also something that Google can control if competitors come along.
i.e. If a reasonable competitor comes along that is willing to sacrifice ad revenue for better search result quality than Google, google can just adjust their search quality upwards to knock them out (and then adjust it back once the competitive threat is gone).
Perverse incentives from Google are all over the place - Searching for the delivery business "Just Eat" in the UK for instance returns an ad for their competitor Deliveroo above the legitimate organic search result for me - and I can also see that JustEat are trying to pay for their own brand name just to compete - and IMO this sort of behaviour is anti-competitive, borderline extortion considering Google is the de-facto way of searching for a business, and shocking from a search-quality perspective (where the wrong result is intentionally shown at the top because they paid more money).
Google's search algorithm is tuned up for searching the whole web. It turns out the heuristics you need are very different depending on the size of the collection.
When Gerard Salton was doing IR experiments with punched cards he was working with collections of as little as 70 documents and in that case you are going to be very concerned about recall and not precision. Maybe there is 1 relevant document and if you miss it you failed.
If you had 70 billion documents you might have 10,000 relevant documents and if you lost 60% of them you still have 4,000 documents. The end user gets more results than they can sift through.
Thus I always groan when I see a site is using "Google Site Search" because the relevance is usually worse than you'd get with the alternatives.
Connected with that is the tuning work: Google has sufficient data to tune up a big model for everybody but true personalized search eludes them because they don't have enough data from you to tune up a model for you.
Why would they do this? Google's customers are the advertisers, not the end-users. And no one is going to pay for a search engine, it's been tried and has failed.
Always curious about things like this. I certainly would pay for this; it sounds like many other people here as well would. I'm curious if the constraint is that there aren't enough people to actually pay for the investment required for the service, or if there aren't enough people willing to pay to meet the standard VC notions of success. We seem to have a problem with building and supplying services for niche (read: "not expressable as an integer percent of the world's population") customer bases, and I'm never sure if that's a business problem or a cultural problem.
Nobody is going to pay for /just/ a search engine. But they might pay for, say, a /better/ search engine, plus additional features around gmail/gcal/gdrive.
Think of it more as subscribing "to google" and less as subscribing to "google search".
Regardless, the point isn't to "fix" google. It's to highlight a possible path for a new market entrant.
... If an existing player wanted to make a move here, I would say that both Mozilla and Apple are well positioned to add "personalized search" to a subscription service. Same with Microsoft. DDG could also make moves here if they expanded beyond search.
A company with an objectively superior search engine could make even more money with ads so now you’re back to the beginning
An “objectively superior” search engine, from an end user’s perspective, might have to make engineering choices that come at the expense of ad revenue.
But we’re all just talking hypothesis, it’d be cool to see someone launch a startup to get some answers.
It's not open source as far I know and there's only the free trial way to try it.
Google could easily use its current fingerprinting to constrain (to an extent) multiple votes. Even knowing only a portion of the population will participate in the voting they can use a Wilson confidence interval[0] or similar to properly weight votes.
Random sampling works here since you're not guaranteed one vote per user per page and the outcome in binomial, seen and downvoted or seen and not downvoted.
[0] https://www.mikulskibartosz.name/wilson-score-in-python-exam...
They're going the OPPOSITE DIRECTION from this!! They recently removed all downvote visibility of YouTube videos from the user, so now downvotes only feed into their algorithm. So in the last line of defense of me ending up watching a shitty video, one of the most valuable tools has been removed by my betters. It's preposterous that people think that Google is doing a good job. They're actively getting worse, and ignoring everyone saying so.
All the big channels I follow are mad at this change, and there's a coordinated effort to bring it back.
you mean the dislike counter they just disabled to force people to sit through more low quality content and pre-roll ads to claim increase in platform engagement and viewership?
The only thing matters is revenue and Google had increases in acquisition costs in prior revenue reports. Expect to see the data points for the latter metrics to be highlighted on the earnings announcement, and a record quarter for YT coming out of the change.
I would love for Google to build this in. Until they do, there is a WebExtension that does this: https://addons.mozilla.org/en-US/firefox/addon/hohser/ ("Block or Highlight Search Engine Results"). I use it to block stuff like W3Schools so when I search for something, MDN is always #1. Saves me a lot of time having to add "MDN" to the end of every query.
Things have changed in the past few years, now that Google has developed advanced transformer models, but for a long time Google's question answering facility has been: "let spammers make 10^8 pages where the title is the question and the answer is in the page".
The trouble is that there's a fine line between "answer is in the page" and "word salad!"
Not convinced this would help. The spammers would just hire people to dislike competitors
> - Allowed you to block certain domains from your search results
This I would use. Never show me results form collider, watchmojo, ranker,
> - Allowed you to create your own custom search engines, like "Programmable Search Engine".
I think this would lead to people writing highly polarized engines. The Red Pill engine for example and we'd have a new problem, the proliferation of popular highly biased results. Of course that's not to say Google's results aren't already biased but they certainly are trying to cover everyone.
Are there any search engines that do this? It's a great, simple idea.
Couple of positions up or down in google results for somewhat popular and valuable keywords can mean the difference in thousands of dollars per day of ad or affiliate revenue. I suspect it would get pretty wild if google launched something like this. There already are black-hat seo methods and services, but something so simple and direct would turn it up to 11.
They already have the tech to fight this on YT. They, in theory, are supposed to be doing the same thing to detect inauthentic behavior on ad placement and click abuse.
It's that Google has destroyed their own search results in order to continue to expand their revenue opportunities.
If Google:
- Enabled downvoting on results, like YT videos. (Has its own spam problems, just like YT)
- Allowed you to block certain domains from your search results, like YT videos. (If they added some kind of "coordinated network detection" and down-ranked domains coordinating with ones you've blocked, that'd be pretty cool).
- Allowed you to create your own custom search engines, like "Programmable Search Engine".
That would be incredibly valuable. They already have most of the tech. They could even create a subscription service around custom search engines if they really wanted. Plenty of people would find something like that incredibly valuable.
Anyhow, buried in there is your startup idea. Remember: your startup doesn't have to generate the same revenue or profit as the incumbent on day one to be successful.