There’s a whole field of research on this (look up Floyd Toole) - while any one individual can have skewed taste, on aggregate people prefer speakers that are as close to neutral as possible.