Well there is a small caveat in this case. The majority of the people doing the mining run on clients that are hosted on just a few github repositories that are run by the main development team. If the developers push changes to these repos implementing the soft/hard fork... yes it is theoretically possible for these miners to reject the changes and continue on mining using the old clients but what is probably going to happen is that practically everyone is going to update to the new client and accept the forks as a patch. This is for the simple reason that most people want to keep their clients up to date and have no strong opinions on the matter. This way the development team can turn every "I don't care" vote into a "fork" vote. The idealization of independent miners is for all practical purposes an illusion.