Which means that you can't have all features when streaming an AAC file, in a time when streaming podcasts is getting more popular every day. You first have to download the whole file before you can display e.g. chapter information.

Of course you could just embed that information in the RSS feed, but you could do the same with jump tables for VBR MP3s. So AAC does not solve all your problems, you just get a different set of problems.

Why not a podcasting specific container that handles streaming and fixed cases, includes support for all the novel stuff podcasters want to do, and so on? Container development is one of the easier parts of multimedia, and there's too much thinking around a specific codec being tied to a specific container (just like your comment). Legacy support is of course the big elephant in the room, but I'd venture a well-founded guess that the average podcast listener is listening on a device that supports apps. (Maybe I'm wrong.)

I can't think of a single good container format. They're all rubbish. Even Matroska is arcane and very poorly documented, and suffers from the end goals and types of media it is used for being encoded in the design. Containers are a giant space in which to innovate and have been that way for years. Unfortunately, open source multimedia development is largely tied up by a certain group of people with a certain background and very specific use cases in mind.

That context from the podcast would have been awesome in this piece. Why wasn't it there?

I also think existing containers would meet the requirements - for example, Ogg meets the requirement of both being storable and easily streamable. Also, the Matroska spec is being updated to a new version, and being standardized in the IETF CELLAR working group. I'd highly suggest participating if you'd like to help improve both the format and the specification.

Matroska draft: https://datatracker.ietf.org/doc/draft-lhomme-cellar-matrosk...