You say "I've met way more people interesting in learning math than performing it", but I have found it downright impossible to learn mathematics without performing it. And I daresay this is true of most people -- a well-known feeling amongst mathematics graduate students is to read a chapter of a text, think we have understood it, then turn to the very first exercise and get completely stumped and have to backtrack.

A lot of mathematics is optimised for pencil efficiency for a reason, and it's not at odds with learning mathematics.