I would argue if you're not bringing your idea to market, that's not innovation. It's the opposite, you're blocking innovation.

Patents stop other people from creating useful stuff. I think it's reasonable that you should have to fill the gap in order to retain the right to stop them. Although enforcing that is difficult.