> Without Google those sites would lose the majority of their visitors

That's not neccessarily true. Without a search engine, those sites would lose the majority of their visitors. But absent Google, another search engine would do the job.

A search engine is a vital component in today's web. "Google search" is not, and its market power dominance doesn't give it a moral blank check.

That's true. However, I fail to see moral wrongdoings by Google in the case of news publishers.

mostly stealing their content in various ways and not compensating them for it

You don’t see an issue with google scraping news contents out of the news articles and displaying a large snippet in the search results that eliminates 90% of the potential visitors?

If they don't want google to display a snippet, then they can "Use the nosnippet meta tag to prevent Google from displaying a snippet for your page in Search results, or use the max-snippet:[number] meta tag to specify the maximum length for your result snippets. You can also prevent certain parts of the page text content from being shown in a snippet by using data-nosnippet". [0]

[0]: https://support.google.com/webmasters/answer/35624?hl=en

That’s playing Google’s game. Where today you “simply” have to do X, which tomorrow will be X plus Y but not Z, and so on. What about Google don’t stealing content by default unless I add a meta tag named yes-google-steal-my-content?

I saw the earliest reference to "nosnippet" in 2007, 13 years ago. Those things are not changing that fast at all.

Did Google change how they do this recently? Because I'm not seeing what you describe. All I see is a headline and an image without any snippet at all.

Just Devil's Advocate, but how would a pure search engine make money without doing what Google does? Let's suppose a hypothetical Google alternative just purely distributes clicks to organic results. Where does the money to pay for its operation come from?

Serious question.

I honestly believe that one of the flaws of the current form of the internet is the need for a search engine.

That's a false dichotomy. As the GP pointed out, Google started with a symbiotic relationship with the websites. Scraping the content and using it to help people find stuff (and profiting handsomely!) is not the problem.

Problem is all other behavior highlighted by GP - pushing organic results below fold, insta-answers totally stopping traffic to sites actually providing that content, and so on.

Google is now using its dominant position in search engine market to effectively bundle search with these other things that negatively impact publishers into an "all or nothing" deal.

"Don't want us to scrape and show your data? You have nice traffic to your page, it'd be a shame if something were to happen to it..."

Users want snippets, they want instant answers tho. Just serving 10-blue links, no summary snippet, that forces you to click through and read the page before deciding if it was relevant to you, costs the user both time and bandwidth. If the article is not what I wanted, then you wasted a few seconds of my time that I will never get back, and if I have a metered internet connection, you forced me to load up a lot of junk, eating into my bill, for naught.

If I want to look up the current time, or current temperature, or how old Trump is, I don't want to have to be directed to his biography page at the Whitehouse.gov, where I then must skim-read several paragraphs to find it.

People are saying "but they're not getting the traffic". Well, facts are not copyrightable, and if all I'm looking for is a small specific fact (answer), then I wasn't interested in a full pageview of your site anyway.

Let's say a put 8 hours into writing a programming blog, on say Kotlin, with tutorials and stuff, and someone searches for a question about "which library contains atomics"?, am I supposed to be mad that it answered back "atomicfu" instead of forcing you to read an entire tutorial on Atomicfu I wrote? If the searcher wanted to read a tutorial on atomics, that's what he would have searched for, and likely, he will go examine this library, and then search further about how to use it.

To me, the future of search engines is an AI, like GPT-3, that goes out and digests the world, and then lets me ask questions, which it will synthesize brief answers to exactly what I want. In this world, the news publishers will face the same problem of demonetization, and people will be asking their GPT-5 bot "can you summarize what happened yesterday around COVID-19 and the Whitehouse?"

News is facing a serous pressure now, but is the right answer to attack summarization technology? It seems it's inevitable that content will be understood and transformed by machines in ways that will be hard to litigate by copyright claims if they spew out original work, and therefore the real question is, in a future of Star Trek like search engines, how can news's basic business model and incentives be transformed?

Personally, I agree with Andrew Yang on this, I think a lot of publishing is going to go open source and non-profit, funded by government grants as a basic necessity of democracy, like healthcare, or universal education.

Google has a search monopoly in many countries. Are you implying that if Google did not exist, no one would visit websites any more, or read the news? They were originally a search engine to find other things on the web, not so much a portal.

No, I'm not. I'm implying that in my eyes it's hypocritical to complain about being indexed by Google with snippets while benefiting hugely from that [0]. Site owners can control what Google indexes and if a snippet shall be displayed [1].

To me it looks like news publishers failed to adapt their business models to the 21st century and now want to milk Google, even though Google brings them a lot of revenue in the form of visitors.

[0] https://www.theverge.com/2014/11/5/7160587/german-publisher-... [1] https://googleblog.blogspot.com/2007/02/robots-exclusion-pro...

Search engine is intermediating and increasing competition.

Without SEO competition for articles, we'd probably have fewer but more profitable news websites instead of thousands of sites praying each other's work (asymmetrically).

It's a mixed bag.

Well, I can't argue against what you're saying in your sub-comment here, Google is a for-profit service and provides traffic to other sites, and it's entirely optional via robots.txt whether people want to participate in being indexed.

The problem enters with the insidious tracking that Google uses, that pushes their revenue per 1000 searches to way above any other search provider, alongside their monopoly position in many countries that allows them to be the default search choice in pretty much every major browser and device that's going to search for anything.

Instead of being the agnostic middle-man as search used to be, Google is encroaching on publishers ability to be sustainable propositions, all the while taking their content.

In the UK, Google has 95% of the search market, and 50% of content discovery is via search (sorry, no source at hand). Basically the choice of publishers is to cut off their arm while content aggregators suck the value from their unique content. It's not exactly an elevator pitch to prove the point, but there's countless parties that have elaborated along these lines.

Surely there is room for a search engine purely to point you to other places on the web, is where I stand. Google serving news is just them providing a reason for you to load up google.com rather than any other place, which clearly they've massively succeeded in doing, being a portal rather than a middle man.

In fact, that's how Google got their initial foot in the proverbial door - by being just a search engine, and presenting a clean landing page with essentially just the search textbox, at the time when incumbents (Yahoo, Altavista, ...) had their landing pages crowded with news and advertisements so much that the actual search box would get lost in there.

Search is really only the main driver of traffic for national sites.

Local and niche coverage get traffic though a mix of social, newsletter subscribers, referrals, and direct visitors.

Without a feudal lord, peasants wouldn't have land to farm. News organisations aren't peasants but the issue is that the Lord is able to control & skim the majority of value from the resource (in this case, attention).

They can choose the block google anytime they want; comparing them to lords and serfs is not a good comparison.

Right, and a serf can technically leave. But they can't, because they lose access to the pertinent resource. Where else can they farm?

I don't like the media, I'm not trying to make out they're weak. My issue is that Google monopolising attention and then unilaterally taking news website's work for free is BS. They're skimming the vast majority of value from the industry with no kickbacks. They get away with it because no inidividual news organisation can fight. The only option is collective action (won't work, too much reward for defecting) or legislation.

Without Google, people would find different ways to reach news content.