Well it's part of math, which physics is already based on.

Whereas metaphysics is, imo, "stuff that's made up and doesn't matter". Probably not the most standard take.

I'm wondering, isn't Information Theory as much part of physics as Thermodynamics is?

Not really. Information theory applies to anything probability applies to, including many situations that aren't "physics" per se. For instance it has a lot to do with algorithms and data as well. I think of it as being at the level of geometry and calculus.

Would you say that Geometry is as much a part of physics as Optics is?