No, "exponentially faster" means exponentially faster with exponentially more hardware. Specifically, N processors can solve a problem of size N in O(log N) wall clock time, where the previous (serial) algorithm used O(N) wall clock time. Most programmers would say "more scalable" or "more parallel" rather than "faster", but the terminology makes sense and is standard in the context of the PRAM model of parallel computation.

I'm not sure, but I doubt this algorithm is "faster" than the previous one on a single processor.

(The article is drivel; I couldn't figure out what they were talking about either, until I looked at the paper)