Not everything in math is suitably expressed (or, perhaps, discussed) in an algorithmic form. Much (most?) of mathematics is operating at a level of abstraction that is far removed from an underlying machine model (in contrast to algorithms, like, well, the cryptographic algorithms you mention).

Take matrix multiplication, imagine seeing this:

  C = zero(m,p)
  for k from 1 to n
    for i from 1 to m
      for j from 1 to p
        C_ij += A_ik * B_kj

Now suppose that B is invertible and we actually want to know A in terms of B and C. Oops, we can't figure it out because we've tied ourselves to this algorithmic expression. In contrast to a more typical algebraic expression which describes not a computation but a relation:

  C = AB
  CB^-1 = ABB^-1
  CB^-1 = A

Perform that manipulation with the algorithmic description that helpfully obfuscates the relationship between the parts.

[And this is a small example, algorithmic expressions of algebraic ideas, like plain English expressions of the same, does not scale very well.]