Yup, lithium batteries at phone/laptop scale should be stupidly cheap.

For an example, go to hobbyking and look up what a 2100mAh 11.1v pack costs. It will be ~$8, and this is something capable of being subjected to 30+amps at 11 volts, not the nice gentle usage we give our phone batteries.