It sounds like what you're describing is roughly the same as the Bonferroni correction: https://en.wikipedia.org/wiki/Bonferroni_correction

It's no panacea. When you use it you reduce your statistical power - in other words, your likelihood of detecting genuine effects goes down.