That’s a perfectly reasonable default fallback, but I think what OP is asking for is a per-site preference the user can specify within the site. Currently you can do this without JS on a single page (with eg the checkbox hack or this CSS Toggles proposal). But you can’t persist it between pages and visits without either JS or a server-sent Cookie header.

All of that said, an aside: if you do implement a per-site setting, it’s a good idea to make it tri-state: light, dark, system. That way the user can remove their per-site setting if they’d prefer to go back to the default.