If you had constructed a new technology that is an improvement of a patented technology, the original owner doesn't gain ownership over your improvement. They only have the right to prohibit use of the original technology. If you patent the improvement, then anyone who wants to use it would have to license both your patent and the base patent's inventions. There isn't a concept of "derivative works" (like in copyright law) where if you didn't license the base patent, you lose ownership over the improvement. If you had failed to patent the improvement, then yes, the patent holder could conceivably use your improvement without paying you, but that's a different scenario.

In very large and complicated fields, you'll often have a minefield of conflicting patent claims, much of which are minor improvement patents. These often coalesce into patent pools, assuming all participants in the pool are interested in playing ball. (Looking at you, H.265.) You don't see certain players looking to invalidate patents on the basis of not having been paid for the right to improve their invention, because such a right doesn't really exist.