In principle R language is equally potent than Mathematica's one (functional+self-modifying), so Mathematica clone can be done in R (and in fact there is a built-in function "D" which calculates derivatives of native expressions symbolically).

Of course this won't ever happen because of the paradigm -- R is for data crunching and does this way better than Mathematica.