This isn't even about safety. It's the pilot's job (if he wants to keep it) to make the passengers as comfortable as possible also. That's why they dodge turbulence. Except in extreme cases, it has nothing to do with safety.