The problem is that often you want to use your smartphone as a client to the same services. This essentially eliminates the second factor.

The problem also is that the smartphone can be hacked and opens a huge attack vector that you don't have with other systems.

Also, typing in stuff by hand is a general bother.