Why? Well, it's handy if a plane can start its own engines, and big engines can't usually be started by batteries, so batteries start the APU and the APU starts the turbines. The APU is also used to drive climate and entertainment systems when on the ground (and not connected to the Ground Power Unit, also mentioned early in the thread).
Finally, the APU can provide fault tolerance: if the turbines are disabled in flight, the APU can provide hydraulic pressure for the control surfaces, as well as electricity for safety systems and the potential to restart the turbines. There is some correlation in turbine failures (flying through something, fire, one turbine blade flies out and destroys a second engine), but the APU is inside and less likely to fail at the same time.
Related is the Ram Air Turbine, which is basically a small wind generator that can pop out of the fuselage if needed to provide some power to critical system when really nothing is working.