Well, to be frank when you provide someone your credentials you oprn yourself to a new world of pain.

But this is also the case when you allow your bank to aggregate info from other banks. At least you can hope that they protect it somehow (because of bank regulations, but this is a very thin hope)