What? Physics is not a branch of mathematics, it just uses mathematics and mathematical properties. That's like saying psychology is a branch of mathematics because it involves statistics. Using math isn't the same as being math. Admittedly physics is one of the "mathiest" fields, but it still involves doing experiments and coming up with mathematical models that fit the data rather than coming up new kinds of mathematics. The line was much more blurred in Newton's time, but these days the distinction is pretty obvious.

Certainly physics is not a branch of mathematics, but it isn't uncommon for papers in theoretical physics to essentially be math papers with an eye toward applications to some physical notions, and physicists do come up with new math.

I would restate this so that software engineering and physics are fields concerned about building practical models. They are similar in a way that they both employ and invent models communicated through a mathematical formalism. "Model" is the key here, not mathematics.