When I look at the end results of work, I come to the same conclusion that indeed it ultimately consists of reconfiguring matter. That's what we need to do to reach whatever goals we have. Maybe the conceptspace work maps to the levels of intricacy of the end-result matter configuration, i.e. the more complex, precise thing you want to get, the more energy you need to expend on thinking rather than arranging atoms. It's easier to move a ton of bricks from one place to another than to design a microprocessor.
So I guess that ultimately I agree with your comment.
(disclaimer: writing this after two beers)
The output of a programmer is the output of their mind, not of their muscles. I've stacked boxes in a warehouse and that sort of work is altogether different from telling transistors when to output a high voltage instead of a low voltage, which is fundamentally what we are doing when we are programming.