There are many more flags such as -ffast-math where the project will be better positioned to decide if they make sense than some random packager juggling hundreds of packages. Even for optimzation flags that don't change behavior there is no clear best set of flags that you should blindly apply to all projects.