I did a security audit on a company that had a setup similar to this.

After popping an internet facing web server, I was able to compromise the IPMI system and use the management network to bounce around to any server in the enterprise completely bypassing all the firewalls and segmentation on the production network.

Management networks need rack level isolation.