IIUC, in a game, let's suppose you pick up an object. We'll ignore the fact that you probably provide some sort of fairly broad movement to indicate that you want to pick it up - the game just notices this, and your on-screen avatar takes care of the rest. But now that you have it in your hand, you do not actually have control over in a direct sense. You may indicate that you want to lift your arm - all fine, but that information will be fed into a physics engine that will work on actual speed & trajectory, obstacles and collisions etc. What is shown on screen is not the result of you dragging a GUI object across the screen, but the indirect result of your indicating "lift my right arm" etc.

By contrast, most design/creation applications give you the ability to point with the mouse (or touchpad or whatever) at some object on the screen and then move it (sometimes along 3 axes). The speed of the motion, the axial extents and the destination come (at least initially) from you. This is true of drawing applications, design applications, DAWs and lots more.