Well, "solid" doesn't have to mean "not a liquid/gas/plasma", it can also mean "all the way through". For instance a table made of "solid oak" is not trying to say it's not a liquid, it's trying to say it's not veneered particle board.

That's how I interpreted the use of "solid" here, but I'm no electronics engineer of course.

Solid capacitors are ones that do not contain liquids. Most electrolytic capacitors use the junction of a metal and a liquid as their active element. Solid ones use the junction of two solids.

"Solid" in electronics means the former (state), not the latter.

It means solid as in the phase of matter in this case. The electrolyte in a tantalum capacitor can either be a solid or liquid.

I don't think that's universally the case, for example solid-state storage refers to the lack of moving parts.

https://en.m.wikipedia.org/wiki/Solid-state_storage

Edit: though it does appear to be the case in this context, just to be clear.