You may be able to find a programming job that's less mentally demanding, to the point that it allows you to reserve most of your mental / creative energy for your own projects. Might take a few tries, but there's probably something out there that would accommodate your interests (depending on where you live, etc).

Best of luck in either case. I can definitely relate to what you're talking about, so no, you're not "just crazy" FWIW.