The bottleneck was with the serial protocol between the drive and C64. It was done via bit banging in software due to some chip bug. The fastloaders replaced the protocol with something more amenable to bitbanging.