There used to be regional/temporal barriers in place before the Web was popular; newspapers had geographically limited distribution and it took time to print a new edition. One newspaper "scooping" another by one day was all it took to get people to buy the one-day-earlier publication. Also, 20th century newspapers collected significant revenue from classified advertising, people buying the paper just to get a weather forecast, and other kinds of information distribution that really didn't have anything to do with investigative news. The Web unbundled all that (weather.gov, Craigslist, etc.) and the only remaining strength of newspapers was producing original reporting. Which, unfortunately, was never all that profitable on its own even before you get to the "facts are not copyrightable" issue that I mentioned in my first paragraph.
Does anyone have any good pages on that which go into how to extract facts without copyright infringement? And for purposes of creating independent, educational works from those facts?
The end result is reputable news sites became clickbait and eventually people stopped caring.
There is still a market for in depth journalism, and we see a rise in that sort of things. There are plenty of youtube channels doing documentary-like videos on current events. That is journalism.
The journalism that is dying is the stuff concentrating on breaking news. If its shallow its outcompeted by twitter. If its higher quality but still racing to the headline, its outcompeted by wikipedia. The fact is the competitors to breaking news journalism are cheaper and higher quality.
Also, I think traditional newspapers should position themselves so they're not competing with the lowest common denominator of basic info, however due to cutbacks, most newspapers are in essence not doing very much in depth journalism anymore, which means they are unfortunately positioning themselves as competing with any other source of news.
So while you could say a decent chunk of the replacement content is worse than what traditional news outlets could offer, it's at least free and exists by the bucket load, meaning the incentive for paying for it is nonexistent for most of the population. Meanwhile the more niche stuff is both free and done better elsewhere, so RIP anyone making money off that anymore.
To make things tougher, getting advertisers to pay for it is becoming a lost cause too, since again, they can advertise in places where metrics and tracking options are better. They'll get way more bang for their buck on social media than they will trying to market on a news site or in a classifieds section or what not.
Today, it is just a bunch of people collecting tweets of random people on the interwebs. Race to publish before competition means there's no time for editorial review of simple things like grammar and coherent thoughts let alone accuracy, so lots of FUD can be spread very quickly as "news".
Those well thought out articles are also considered too long and boring and get reposted on socials as TL;DR as if it were their own thoughts.
We used to make fun of the microwave generation with "I want it now" type comments. Now, it's I need it in less than 140chars, or I'm scrolling past it.
The death spiral you describe is like a train wreck that you can do nothing about.
News has become a weird kind of curation market. And the funny thing about this payment arrangement that publishers want is that they are not paying any of those tweeters who may be breaking the news or themselves curating the info.
This was only desired and seen as useful and interesting when people had nothing else to do.
Everyone loves reading stories that confirm everything they already know.
Big real news spreads like wildfire, but news as it is consumed is entertainment.
I mean, seriously, that's the reason.
Back when newspapers were healthy, basically there was all this stuff that was bundled with the actual news that was of more immediate utility to people: classifieds, movie reviews, comics, coupons. And there was less competition for people's leisure time.
News is now in a very competitive leisure time landscape and is debundled from stuff that's of high value, and most people just don't care that much about news. They'll read it when it's around but they won't pay for it. The ones who are willing to pay for it mostly just subscribe to the New York Times because in a nationalized news environment why not go for the biggest producer.
News provied two services in the old days: access and filtering.
Back in the day you couldn't easily get things straight from the horses mouth, now you can just go to their website.
If you did have direct access in the old days, it was all way too much. There was no way to filter to 10-minutes worth of top goings on. Now a days you can just look at what is upvoted and stop once you have read enough.
I stopped caring about what was happening on the world stage in 2022, and it has been sublime. I hear about the big things from friends and random folks, and it gives some instant conversation starters (“oh no, I hadn’t heard about that”). The only news I actually care about is local news, and I skim the newsletter my (small) city publishes on Fridays.
Given that, there is zero value provided to me by traditional media publishers. I say this as someone who gets NYT, WSJ, WaPo, and FT for free through work. I don’t use them, because I’d rather spend my time doing something else.
No judgement here though—if reading the paper is your thing and you like staying informed, that’s great and I fully support it. Just offering my perspective.
Add to this the introduction of terrible news media. The spam and click bait media, where you get the articles for free but the articles you get are basically worse than what you’d find anywhere else because they are most there to generate revenue from advertising. Stuff that gives “traditional” news media a very bad image. Further add the media platforms which are now straight up political propaganda, which some always where, but now they are owned by very few people and push very divisive agendas which is a little different from the past where they were more inline with each other and the “general” aristocracy rather than a few billionaires/oligarchs and you gain the general media perception even more. Both are rightfully so.
In a sense you get what you pay for though. If you’re not paying you’re not getting quality news. You can see this in Danish news very clearly, where only a handful of news papers still do actual critical journalism and the rest mainly do opinions or click-bait articles on current events that you might as well read on Facebook. Not because Facebook does anything particularly in that department but because the contractors who sell stories and pictures to the “free” news media typically also post their content elsewhere for increased revenue on their part.
As far as going to the “biggest” most “cultural” news media I totally agree with you. In my country people read Weekend Avisen, and then add one of the more politically inclined subscription papers based on their views. But basically it boils down to thee papers, one that is slightly more conservative (which in Scandinavian/American optics would make it almost socialist), one that is left-leaning (again, very socialist in American optics) and one that is based on Christianity (but extremely moderate, as in pro abortion and gay marriage).
Aside from that there are two “localised” papers which either focus heavily on Copenhagen or the part of our country which is called Jylland.
Almost none of these papers would be alive without government subsidies. Because the only two papers which actually makes money are Weekendavisen (our NYT) and the Christian paper which doesn’t actually make money but is subsidised by various Christian groups.
Plenty of independent journalists are out there doing just fine and even some of the big outlets have adjusted effectively. But in the internet age we don't need hundreds of outlets all reporting on the same stories 24/7 - especially when most of those stories have no impact on the viewers/readers.
The market is simply adjusting and the old establishment is going down kicking and screaming.
For example, Netflix is $84/year. Netflix content seems a lot more valuable then simply twice what WaPo provides.
They'd rather watch TV, and not even the best TV. If you want the best, you need to pay for Max and Disney and Apple and Amazon and the rest.
Besides, if you actually like to read why would you be buying washington post? People who like to read usually aren't going into 5 minute long shallow articles, they are getting a book on the topic.
Either way, producing quality journalism costs money. And I’m ok paying a reasonable fee for it. :shrug:
It's the news sites' content that Google is scraping and monetizing. How is it not Google that's latched on to a "free teat"?
Serious question. What am I missing?
EDIT: Thanks for the downvotes everyone. I need 'em from time-to-time to ensure I've not succumbed to The Matrix.
Of course, you're all wrong. But, keep 'em coming!
The news outlet can use robots.txt to prevent indexing. If Google doesn't bring them value, there's the easy answer.
The article is part of the overall content that Google displays in its search results. And, of course, Google monetizes its search results with ads.
I honestly could care less about either of them. I think they should just fight it out on their own and keep our legal system and tax payers time and money out of it.
There's definitely some symbiosis here, but it's ultimately Google that's dependent on the news (and other) sites' content, which it gets for free. That is, the news sites (and other content providers) could exist without Google. But, Google could not exist without their content.
At least that's my observation. So, I wasn't picking a side as much as earnestly asking how OP concluded that its the news sites wanting something free from Google versus the other way around.
Of course, that's the way it is. But, is it a good thing that they've intermediated all of the world's content?
It's easy to argue that the problem is exactly that a relative handful of sites have a monopoly on traffic and, what's more, they've gained that monopoly for free.
But they don't do that, obviously, because they depend on Google to send them traffic.
30 seconds of serious thought would tell you that your observation is wrong.
Again, a news organization can just change their robots.txt to block google from indexing their site.
They don't do that because that would instantly kill all their search traffic... and most likely kill their business.
If CNN changed their robots.txt to stop being indexed by Google, Google would literally lose 0 users.
> I wasn't picking a side as much as earnestly asking how OP concluded that its the news sites wanting something free from Google versus the other way around.
It's been explained to you several times. Instead you're more interested in acting self-righteous (it's honestly pretty cringeworthy).
Or, maybe I just have a different opinion.
>news organization can just change their robots.txt to block google from indexing their site
You don't seem to have thought beyond this superficial robots.txt "solution". Yes, we all know that option is available. But, as I've self-righteously offered for consideration, Google is one of a few sites that essentially monopolizes traffic generation, so they've positioned themselves to make it untenable for sites to block Google's crawling (and free monetization) of their content.
Cory Doctorow has (another) recent Twitter thread on how tech monopolies have grown virtually unchecked and now abuse the ecosystems in which they operate, increasingly clawing back more value for themselves at the expense of others. IMO this fits the pattern. Look it up. You might find it interesting. Or not.
>They don't do that because that would instantly kill all their search traffic...and most likely kill their business.
And there you've just stated exactly the problem I'm referencing, with apparently zero awareness of how someone could find it problematic. I mean, you just said "they could solve the problem by blocking Google via robots.txt, but that would kill their business".
So, not exactly a solution then, right?
It's baffling that you can say this but still angrily scream that "It's robots.txt! Case closed!"
>It's been explained to you several times. Instead you're more interested in acting self-righteous (it's honestly pretty cringeworthy).
You clearly don't hear yourself. Calm down.
EDIT: out of curiosity, I just took a quick look at your recent comments to others. One of the first to pop up was this:
>What are you even talking about? That's not how SEO works in the slightest....?
Lol am I still reading HN or is this Reddit?
Yeah, no self-righteous cringe there.
Sure, but these things are directly related. It's the scraping that generates the links in question and ultimately earns Google ad revenue. And, if they did pay on the scraping side, then charging for linking would be less relevant. As it is, they're collecting the content, but not paying on either end.
So, it seems if you agree they should pay for the scraping (but they are not), then you wouldn't be opposed to them paying on the other end. In fact, this might be fairer to Google b/c it's pay for performance.
But, more to the point, I was responding to OP's specific-claim that the news sites were attempting to "latch on to a free teat".
Understood. I use the term "scraping" loosely and, admittedly, purposely. I think it's illustrative of the broader point I was attempting to make.
But, if you really want to be precise, what Google actually does is more commonly (and euphemistically) referred to as "crawling". And, it is more accurate to say that they are crawling for the purpose of indexing what they've crawled. Crawling is essentially the front end of an overall process which ends with indexed results.
Whatever the nomenclature you prefer, the effect is the same, and so is my point.
>websites go out of their way to encourage google
I understand that some websites "encourage" Google, and that intersects with the alternative point of view I've been suggesting. That is, that Google is monopolistic in its traffic ownership and the content owners have little choice but to offer their content to be freely monetized by Google. It's also worth pointing out that there are some businesses which are built around SEO from the ground up while others—like news outlets—pre-existed Google but now rely on them to survive. These are different.
To further close the loop, my suggestion was that the original topic of this thread might also be seen as somewhat of a remedy for that effect. And, frankly, it's strange to me that you will allow that Google should be paying sites for their scraping or crawling or whatever, but don't seem to be connecting it to my point, when it really could be viewed as an alternative remedy to the problem I'm describing.
Overall, I believe mine is a more interesting and accurate way to look at the problem than to simply accept that Google has intermediated so many content sites and their consumers as some natural and universally right state of affairs.
In any case, I don't think Google's search business model or that SEO is a thing, etc. is lost on anyone on HN. And, thought I might encounter more interesting discussion here around my view. But it seems most people here have accepted that Google just owns the traffic and everybody must play along. Further, that it's really for their own good. I suppose it's become harder to imagine a world where content producers own their content and are not coerced into giving it away for need of traffic from a single monopolistic source without whom their business might not survive.
But, it's somewhat surprising when I zoom out and think about the spirit of "hacking" and the audience that used to more predominantly frequent HN. Thinking back to staunch support of folks like Aaron Swartz and other topics. Maybe I'm the only one who sees these as somewhere along the same continuum. And that's fine.
Nonetheless, I find this discussion tedious and boring by now, as I'm sure others do my "alternative perspective". So, let's just agree to disagree, rather than have these pedantic restatements of definitions and Google's well-known search business model as if these are somehow dispositive.
Thanks for your time.
This is a good comment that gets at why[1], TL;DR: the hedge fund thing is a complete nonsequitur. The programs are the equivalent of Google Cloud grants, and Google actively disinvested from News[2] just because that was an easy place to get your mandated Sundar cuts for Wall Street.
They're not a good steward of anything other than their stock price.
I don't like the idea of a link tax but I do know, 100%, that blog post is slanted and mealy-mouthed on everything I know first-hand about.
[1] https://news.ycombinator.com/item?id=40015572
[2] https://www.nytimes.com/2024/02/05/technology/google-layoffs...
All that cash flowing into MTV is real people’s jobs and livelihood being drained away - it’s also the reason search is entering a utility collapse curve since AI can generate unlimited click bait.
Double click will end up killing google, it’s already too late to go back.
Advertisers like that.
It's tough to sell them on something else, and essentially impossible to replicate as is.
The search antitrust suit was garbage, but the coming trials in the ad auction suit is going to hurt a lot more. They total lost their way, it’s actually insane when you’ll see what theyve been doing.
Google never stopped “innovating”, they just shifted their innovation to fraud.
https://adtechexplained.com/google-project-bernanke-explaine...
Alleges quite a bit, yet the main four in the article are.
1) Google says it runs a Second Price auction, but really runs a Third Price auction, ignores the 2nd bid, and takes the difference for (shifting purposes). Telling a Publisher they made $8, when they would have made $12.80.
2) Google inflates bids of buyers "to ensure their advertisers beat out bids from competing buying platforms". Artificially making their platform look better economically. Was individual cases, yet now alleged to be a global pool of stolen Publisher money.
3) Gaining access to Publisher's ad-spend behavior with Dynamic Allocation, and then using the info to make sure rival platforms seemed less competitive by manipulating and inflating other bids. Make sure they always seem to lose by $0.01 on every bid.
4) Forcing Publishers to accept Dynamic Allocation by punishing them with the maximum allowed revenue drop for Bernake (40%) while inflating competitors. Notably Machiavellian that there was a known allowed revenue drop. "We're bein nice, we'll only cut you 25%"
On a slightly different topic, made me wonder how many investment firms artificially lower rates on funds to scrape a bit of profit off the top.
[1] https://texasattorneygeneral.gov/sites/default/files/images/... (note, the child-support location is really weird)
That seems an argument for news publishers rolling their own, not against.
What you are allegeding is nasty, but it should also make it easier to build a coompeting service not harder.
It’s impossible to roll your own for a very long list of reasons that go back to the late 90s.
It's not the world/Google that changed.
But there is competition in this space.
...at the end of the day, I'd be happy to see a tax pass, some mix of what you put excellently as "real people’s jobs and livelihood being drained away", and I imagine similar to you given your MTV reference, experiencing there wasn't really anything special going on at BigCo. It's plain old economic inefficiency, not poor beleaguered good guy G that's always looking out for news and just trying to organize the world's information.
Journalists aren’t dumb either, talk to anybody who directly got one or was involved in the process.
If you're profiting from it, it's not a bad idea.
If they don't want a Google to index them they can use robots.txt to prevent it.
What's actually happening here is that news media orgs were in a huge bubble because of the advent of the internet, and they've failed to monetize effectively. So they're looking for a revenue source to shore up their failed model, and they know they'd lose revenue if they removed themselves from search results.
Possibly perhaps, the big publishers (with deep pockets and lobbyists) really like this approach because despite entrenching big tech, it erects formidable barriers to entry for new upstart competitors to them? Regulatory capture is a tried and true tactic for industries that reach a certain size.
IMHO society is a lot better off trying to prevent summarization efforts (that eliminate the need for the user to visit the page) than they are trying to charge for links. The latter is something that benefits everyone regardless of size. I'm not necessarily in favor of doing that either (would need to think it through a lot more, because it would essentially legislate a worse user experience, which is not something to be done without serious deliberation), but it seems like a much more relevant fight to have and one that isn't so self-serving.
The text: https://legiscan.com/CA/text/AB886/id/2832517
So, at least in theory, some nimble startup could come in and have a lasting advantage against the big existing platforms until they got big enough that they could handle dealing with the regulations.
I do have mixed feelings about the recent laws written like this. It feels like if it's important that we pay news publishers, that should apply to everyone.
It's a different question when there's an excerpt or machine-generated summary. In that case, copyright applies, and in most jurisdictions it may or may not be fair use depending on what is copied and how it is presented.
This looks like a money grab to me. What am I missing?
Google news+facebook+twitter can drastically change the revenue of any online news site with an internal decision. Where else do we see something like this in the economy?
EDIT: the criteria of zero business relationship between your entire product class and the Big Business is critical; otherwise lots of examples exist.
There isn't a single physical product that's not beholden to the Walmarts of the world to stock, market and highlight those products. There isn't a single farmer in this world that isn't dependant on stores buying off their produce and putting it into shelves.
Heck, even in paper era, there wasn't a single paper not beholden to kiosks and other stores to put their papers into racks and into premium places where customers are most likely to pick them up. Do you hink NYTimes demanded that street vendors pay them for the privilege of putting their paper onto a rack?!
Having fully vertically integrated bussiness (like you're mentioning) is a very modern development of monopolies.
In the Google News case, no news sites sells their news to Google News. There is no business relation between Google News or any de jure contact whatsoever between the two. It's different. I am not saying its good or bad, or that this gives the right to the news sites for news sites to demand payments from Google.
But I think its a weird scenario that doesn't occur in other markets. And to understand the pathologies of the online news business, we need to think about this particular no-relations fact.
Also, any business whose livelihood depends on a large company's platform. Businesses and entrepreneurs selling their work on Amazon and eBay, YouTubers and Twitch streamers making content for those platforms, influencers in general given their reliance on social media services...
If I have a product page on Facebook, I have a contract with Facebook. Facebook could choose to end that contract with me and close my account. But Google News blocking/downranking a news site does not involve any contract ending between Google News and the news site. If I don't have a product page on Facebook, Facebook could still choose to block/downrank posts where users on their own free will mention my product. But I don't think this really happens at any significant scale yet.
Same applies your Amazon/Ebay/Youtube/Twitch examples.
Going back not as far, CNN was dependent on your local cable company to deliver their signal to homes. Charter or Comcast could choose to stop carrying a channel altogether.
Yelp.
TripAdvisor.
Manufacturers and Retailers
There are lots of examples of people who produce things being beholden to a few companies that controls the distribution channels.
A more interesting one is small businesses and payment processors which don't control the distribution but rather keep them operationally dependant on a product they control a monopoly over.
Walmart is free to ban Google from the premises (robots.txt) or to negotiate an exchange (monetization agreement).
No matter how hard you try to avoid it, some asshole will always post any video you make to YouTube immediately and probably try to monetize it.
Given how much money Google collects for YouTube, they should get fined through the roof when they allow somebody to upload something they don't have the rights to.
The porn companies showed that you solve this technically. However, Google will fight this to the very end.
Does it? If I summarize a book, do I need to pay a fee? News (much like recipes) is not directly covered by copyright (facts cannot be copyrighted). Only the expression (exact wording) is (which is one reason why recipes are usually accompanied by a personal story).
I see this claimed often but it doesn't make any sense - anyone can still copy the recipe without the story and then the copy would even provide more value than the original.
Seems to me padding recipes with fluff is mostly about SEO and maximizing ad impressions.
On the contrary, it's the for-profit content creators and to a much bigger extend the monetizing platforms that have latched on to the open web and are now trying to redefine it. Meanwhile, most content is still created by users like you and me who are not going to get any of the ad profits these platforms are crying about.
Google has been at the helm of the web for at least 15 years. “Before Google came along” was before the consolidation of the internet. Google consolidated the internet behind Search, their browser, SEO, even Amp pages (remember that dumpster fire? So open, right?), as they slowly tolled more and more of the internet, while keeping traffic flowing to their advertisers.
Content creators are entitled to a cut of the profits that Google makes from selling ads while presenting their content. I hope this law passes and every other state follows suit.
It does get fuzzier if you're also summarizing, and there's clearly some sort of spectrum from "just the URL" to "AI synopsis of the entire article". But at the level we see on Google News (headline + maybe a picture), I don't feel there's any good justification for charging.