Battery installations are still relatively expensive right for the capacity they offer right?

That's a very general statement, but solar plus lithium battery storage is now competitive with gas peaker plants and those can grid balance as an extra service.

That is true, but prices have been dropping rapidly.

Inertia is slightly different to supply response and most inverters use grid coupling for frequency and phase synchronisation so are not suitable. The slow response of old plants is an asset in this case and only a problem if they are generating rather than just providing inertia, which is instant.