I totally agree, and in fact in big scientific code this is an ongoing reproducibility issue---different platforms (like GPUs) can have subtly different floating-point behavior, and how do you know the results hold up.

That said, your floating-point code probably didn't come with a numerical analysis to begin with, so how do you know it's better than the replacement? (Well... Herbie does dumb stuff sometimes, so do use with care.)