It’s generally not trademark infringement to refer to the actual thing the trademark refers to (”nominative use”). This would actually be a claim unlikely to succeed unless you are intentionally claiming that are writing the true original Harry Potter stories.