I'm skeptical that a person can really understand why x^(2/3) is not differentiable at x=0 but has an absolute minimum at x=0 without understanding the algebra. I think one can get an idea of the concepts without knowing algebra but I doubt one can understand calculus without knowing algebra.

We always teach the meaning of a given mathematical concept before doing using them or doing calculations. It's just that true understanding comes from the doing the calculation part.