Norman Wildberger (among others) has put a lot of effort into building up the foundations of mathematics using only the rational numbers and it is quite interesting how far he has gone. He's considered to be a crank by many, but you have to appreciate his effort.
To go back to your original question, his Math Foundations video series, especially #14, #95, and #96 might help.
I'm a bit confused... it sounded like you had objections to defining numbers with infinite decimal expansions, but you're okay with e, pi, etc? Or did you just object to uncomputable reals?
Also I don't know much about this whole topic, but I was taught that you can't fill a continuous line with just the rational numbers – you need the "undefinable" ones for that... What do you think about this?
Also thanks for the references, I might check them out to see what Wildeberger has to say.