You're right, from everything I've read, but there are two other interesting data points:

(1) The idea of the "midlist novel" or "paperback original" basically disappeared for a couple decades -- these are the old mass market paperbacks that you used to see all the time, about 4.25" by 7", that you almost never see anymore. (So, there was a kind of discounting: softcover books were a lot cheaper, even when adjusted for inflation.) There were authors who made a good living pumping out these midlist books at the rate of one or even two a year. The self-publishing boom has brought this back to a degree as ebook originals, although I've talked to more than a few ebook-first indie authors who insist they need to get out four or more books a year to make a living, so it's arguably harder for most. And of course that "most" is "most of those who manage to make a living that way," which is, well, not actually most!

(2) Short story rates used to be much, much higher than they are now when adjusted for inflation, to the point where there were people who made a successful living selling primarily -- or even exclusively! -- short fiction. I've never been able to get a good read, pun intended, on what happened here, other than a nebulous sense that readers' tastes just changed over the years (the "fewer other demands on attention" you mention was likely a big part of that), and those markets became less viable.

Short fiction was being bought by magazines with large reader bases. Magazines have essentially died as a medium over the last twenty years, and fiction magazines were on their way out well before then.

If you've got a larger reader base and lots of competition, you can pay a lot for content. If you don't, you can't. The various TV subscription services are playing the same game that the sci-fi magazines used to; they pay a huge amount to produce content for recurring revenue, in fairly tight competition with the other streaming services to have the best stuff. (Think the expanse vs the mandalorian vs unbounded quantities of star trek.) The primary medium for consuming sci-fi changed as it went more mainstream, but also magazines died generally.

In SF, at least, the magazine ecosystem associated with short stories has taken a pretty big hit which means new authors tend to not get into the genre that way. Of course, that's a bit self-referential because "Why did that ecosystem largely go away?" and the answer is that I'm not sure. Though I'll note that a fair bit is online these days so maybe new authors felt that was a better way to build their name.

I'd also note that some of the better SF short story writers these days tend to write in a mix of genres and often publish in places like The New Yorker.

For fiction publishers were small houses with semi-amateur owners. They had an interest in what they were publishing, and if they liked an author they'd provide opportunities and invest in a career.

For example Penguin, which was launched in the 30s to provide cheap literary paperbacks for the mass market - a kind of cultural levelling up instead of dumbing down.

Now publishing houses are relatively small departments in unimaginably huge media corporations. Penguin is now part of Penguin Random House which is part of Bertelsmann, which also owns BMG (Bertelsmann Music), RTL TV/Radio in Europe, and Arvato, which is a general purpose corporate offering logistics, finance, IT.

So it's not a family-owned business any more. And it is much more business than family, with the usual MBA culture of targets, ROI, and the rest.

>People probably read more books. There were fewer other demands on attention, whether YouTube, social media, online content generally, etc. I certainly read books far less than I used to.

I'd imagine the number of books sold per year is strictly increasing.

If it's increasing because of a growing number of readers, then that's a winner take all scenario where Harry Potter sells more and more copies as each new reader hasn't read it yet.

If it's growing because of one extremely voracious reader buying up every book they can get their hands on, that's a scenario that favours more obscure authors.

Closer to the later for most authors. Though every few dozen years there is another Harry Potter that everyone in the world buys and reads. For most you need to target those voracious readers and what they are willing to pay for - but be ever on the lookout as to how you can jump to the Harry Potter world where everyone buys your books.

Harry Potter was good (in the first few anyway), but if you like that type of thing there are ton of much better books that never made it.

Please name one.

If you particularly wanted books that "didn't make it", I don't know anything about that. But maybe you just wanted books that are like HP but better than HP.

I read the first few HP, and thought they were dreadful, and thus never read the later ones, so maybe I'm not the person you want advice from, but here are some recommendations of novels/novelists in the same genre (fantasy novels, written for children, that hold up for adults):

Nearly anything by Dianna Wynne Jones, but I particularly enjoyed The Lives of Christopher Chant, Archer's Goon, and of course Howl's Moving Castle.

Susan Cooper's famous Dark is Rising series. Half the series is more normal-kid (starting with Greenwitch), half is more special-magic-kid (starting with The Dark is Rising).

Garth Nix's Old Kingdom, starting with Sabriel.

While China Mieville is very much not a children's author (really! don't buy a random mieville book for your young niece/nephew, really don't!), Un-Lun-Dun is an amazing book in this genre.

Thanks for the answer!

I enjoyed the fableHaven series. But YMMV, tastes are different.

This suggests otherwise. (Although this is obviously not a complete set of data. I'd actually probably have expected a bigger falloff but maybe ease of acquisition leads to more people buying books they don't end up reading.)

https://ideas.bkconnection.com/10-awful-truths-about-publish...

I suppose the assumption is predicated on a rising population.

Also you'd think last year might have led to more people reading books.