When I look at the end results of work, I come to the same conclusion that indeed it ultimately consists of reconfiguring matter. That's what we need to do to reach whatever goals we have. Maybe the conceptspace work maps to the levels of intricacy of the end-result matter configuration, i.e. the more complex, precise thing you want to get, the more energy you need to expend on thinking rather than arranging atoms. It's easier to move a ton of bricks from one place to another than to design a microprocessor.
So I guess that ultimately I agree with your comment.
(disclaimer: writing this after two beers)
The output of a programmer is the output of their mind, not of their muscles. I've stacked boxes in a warehouse and that sort of work is altogether different from telling transistors when to output a high voltage instead of a low voltage, which is fundamentally what we are doing when we are programming.
"Work is of two kinds: first, altering the position of matter at or near the earth's surface relatively to other such matter; second, telling other people to do so. The first kind is unpleasant and ill paid; the second is pleasant and highly paid."
I've always liked that quote.