Could it be that the specialization you are working with ends up actually be a benefit with regards to finding new hires?

From my perspective, I'm probably not going to go out of my way to apply to a typical startup unless there is some reason to believe the renumeration is remarkable. However, I might be interested in applying to a company like yours as a challenge to myself.

That happens. For example, to be a low level 3d engine programmer is a quite specific skill, yet because of the nature of the games industry, such people don't get paid as much as your average full stack web developer.

Yes, I used to work in the security field and almost every project was an adventure, a challenge where you also need to learn at the speed of light.