They are pretty slow even on GPU. The problem is that it's an autoregressive model. So it needs to do a forward pass for each token.