If you are going to apply 3 angles directly, then you run into problems (specifically, if you pitch +/- 90 degrees, then roll and yaw become the same thing, aka gimbal lock). If you take those 3 angles and convert them to a matrix, and use that matrix to apply the angles, you're all good. You can then even take the new matrix and pull 3 angles out of that.
But yes, otherwise I agree with you.