Certainly. The RPI is not able to even remotely saturate its network card + a USB harddrive.
Especially with an encrypted disk.
After googling for alternative arm credit card boards I'm now looking into not too old [thin]m-itx atom ones. I wonder what's the cheapest out there.
you will notice a serious performance upgrade by switching to a BeagleBone black. IMHO the BBB is about twice as fast (or more) as the rPi.
The BBB is lacking in community support vs the rPi.
The Pi's processor is equivalent to a 300MHz Pentium 2, and that's kinds scary.