Yes, there are tons of papers on doing optimal scheduling/instruction selection/register allocation by using integer linear programming.

Modeling is not even the hard part, it's modeling in a way that doesn't take till the heat death of the sun to solve ;-)