I think you can. I don't think copyright covers gameplay mechanisms, it just covers the actual text of the rules. But IANAL.
The problem is making a whole spellbook with a one-to-one correspondence to WoTC's spellbook, even if each of those entries has been modified. The spellbook is more than just a bunch of separate spells - it's a collection carefully designed for balanced gameplay and has protection in itself. See [1] for one way that "compilations" can be treated as having more rights.
The EU also allows copyright of databases [2], and I think the spell list meets the criteria.
And don't forget you'd also be doing the same thing with races, classes, equipment, monsters, conditions, deities, feats, blah blah blah. Put together I think it's pretty likely to be infringing somewhere! Certainly enough that a lawsuit isn't getting dismissed.
[1] https://www.copyright.gov/circs/circ14.pdf [2] https://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=CELEX...
My understanding is that this is where things get tricky: realistically WotC probably can't go after anyone that creates an ability that is mechanically identical to DnD ability. Things like spell names, might be, but "fireball" is far too general to be copyrighted. But what about a spell called "fireball" that also has the same mechanics? That's where things can get complicated.
So what if they just doubled all HP values and damage values, and changed any names that are not unambiguously generic ("invisibility", "fireball", etc.) ? Damage values no longer overlap, and spell names are different. On what basis does WotC have to claim infringement?