Does that mean you could rewrite, say, Harry Potter from scratch without violating copyright?

Yes, but unless you change names it would be a trademark violation.

Normally that's the case even though at least one of those books was taken down based on (rather dubious imo) violation of rights on derivative works. It was never a verbatim rewrite nor a trademark violation. The Dutch court even proceeded to make a list of similar ideas between two books which I think is explicitly disallowed under the US version of the copyright law.

https://en.m.wikipedia.org/wiki/Tanya_Grotter

( Not only that, even what they say it's russian culture and folklore, it's based on polish history and tales: https://en.m.wikipedia.org/wiki/Łysa_Góra )

Yes. There are numerous fantasy authors who make a living rewriting Lord of the Rings. Copyright protects the expression, not the ideas.