Quite a few of these can be set in the HTTP headers as well, but people editing templates don't always have access or know how to do it.

BTW; not all UTF-8 text has a byte-order mark, I would say most of them don't.

BOM is not valid in UTF-8 for one thing.

To be pedantic, the Unicode standard disrecommends the use of a BOM in UTF-8-encoded documents rather than declaring it invalid.

It makes zero sense to specify a byte order for an encoding in which it is irrelevant. It only persists because of a lazy vendor that can't encode Unicode correctly.

It would have been nice if every well-encoded Unicode document started with BOM and every legacy doc did not, instead of having to guess whether a doc is more likely UTF-8 or Latin-1.

Then concatenating to valid Unicode documents would no longer be valid Unicode. That is bad. And ASCII text would no longer be a valid UTF-8 encoded Unicode document. That is bad. And even when everything has finally switched to UTF-8 every tool ever will still need to handle the BOM. That is bad.

Guessing between valid UTF-8 and Latin-1 is only ever ambiguous when there are multiple non-ASCII characters in a row and all those sequences are made up of a lead byte with the correct number of trailing bytes. How often is that a problem for you in practice?

If you can't set the header, sure, put it in the head.

What I don't get is why so many say it is required in the head, even when you manage to set the header.

Probably nice for people that save the HTML document locally via curl or similar.

curl or similar could inject a meta tag.

Or we could have a default of UTF-8.

I don't dispute that it can be nice. I am just curious why so many say it is required.

I imagine it's the cumulative amount of my first example, curl, and probably other "save as html" functionality in some browsers, crawlers, s3 buckets, archival tools, and so on. I'm sure many can do as you suggested and inject the header, but suspect many don't.

Another use case might be content that is authored in a content management system. Perhaps it's the easiest way to carry your intended charset to the destination.